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Section  1.0  INTRODUCTION 


Proper  documentation  Is  an  essential 
part  of  the  software  development  pro¬ 
cess.  Computer  program  documentation  Is 
often  Inadequate  because  It  Is  too  brief 
or  because  It  falls  to  satisfy  the  pur¬ 
pose  for  which  It  was  Intended.  On  the 
other  hand.  It  may  be  so  prolific  that 
the  intended  user  is  overwhelmed  by  the 
magnitude  of  material,  much  of  which  is 
not  needed  but  still  purchased  by  the 
Air  Force  at  high  cost. 

1.1  PURPOSE 

It  is  the  purpose  of  this  guidebook  to 
describe  the  required  documents  and  the 
needs  they  fulfill  and  to  provide  guide¬ 
lines  for  the  acquisition  engineering 
process,  performed  by  the  Air  Force, 
associated  with  computer  program  docu¬ 
mentation  for  automatic  test  equipment 
(ATE)  and  trainer  simulators  (TS). 

1.2  SCOPE 

This  Software  Acquisition  Engineering 
(SAE)  guidebook  is  one  of  the  guidebook 
series  related  to  ATE  and  TS  ground  sys¬ 
tems.  The  guidebook  titles  in  the  series 
are  as  follows: 

Software  Cost  Measuring  and  Reporting 

Requirement  Specifications 

Contracting  for  Software  Acquisition 

Software  Statement  of  Work  (SOW)  and 
Requests  for  Proposal  (RFP) 

Regulations,  Specifications  and  Stan¬ 
dards 

Measuring  and  Reporting  Software  Sta¬ 
tus 

Computer  Program  Documentation 
Requirements 


Software  Quality  Assurance 
Verification 

Validation  and  Certification 
Computer  Program  Maintenance 
Software  Configuration  Management 
Reviews  and  Audits 

Management  Reporting  by  the  Software 
Director 

This  guidebook  identifies  documentation 
requirements  and  summarize  the  require¬ 
ments  for  documentation  found  in  Air 
Force  regulations  and  specifications  and 
augments  these  with  Air  Force  and  indus¬ 
try  experience.  The  acquisition  engineer¬ 
ing  process  for  computer  program  docu¬ 
mentation  is  defined.  The  description  of 
the  acquisition  engineering  tasks  for 
computer  program  documentation  makes  up 
the  main  body  of  the  text. 

ATE  and  TS  documentation  is  traced  from 
the  Required  Operational  Capability 
(ROC),  through  the  pre-contract  planning 
documents  prepared  by  the  Air  Force,  to 
the  documents  prepared  by  the  contractor 
for  software  development. 

The  guidebook  is  written  for  managers 
and  engineering  personnel  responsible 
for  the  acquisition  of  computer  program 
documentation.  It  describes  the  engineer¬ 
ing  tasks  required  in  the  acquisition, 
review  and  use  of  the  software  docu¬ 
ments. 

1.3  TS  AND  ATE  OVERVIEW 

This  section  provides  a  brief  sketch  of 
TS  and  ATE  system  characteristics, 
including  the  function  of  the  computer 
programs  associated  with  each. 
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1.3.1  TS  System  Characteristics 

The  TS  system  Is  a  combination  of  spe¬ 
cialized  hardware,  computing  equipment, 
and  software  designed  to  provide  a  syn¬ 
thetic  flight  and/or  tactics  environment 
in  which  aircrews  learn,  develop  and 
improve  the  techniques  associated  with 
their  individual  tasks  in  an  operational 
weapon  system.  In  many  cases,  visual 
and/or  motion  systems  may  be  included. 
Figure  1.3-1  depicts  a  typical  TS  system 
which  employs  digital  processing 
capability. 

The  computation  system,  integral  to  the 
crew  training  simulator,  consists  of  one 
or  more  general  purpose  computers.  The 
computing  hardware  consists  of  machines 
with  hardware  floating  point  arithmetic 
and  sufficient  word  size  and  memory  to 
provide  efficient  use  of  the  simulator 
Higher  Order  Language  (HOL)  language. 

When  a  multi-processor /multi-computer 
system  is  used,  it  must  be  designed  such 
that  all  computers  operate  in  parallel 
in  real-time  and  are  controlled  and  time 
synchronized  from  a  single  computer  pro¬ 
gram  supervisor/  executive.  The  execu¬ 
tive  directs  the  program  execution  and 
establishes  priorities. 

The  simulator  (1)  accepts  control  inputs 
from  the  trainee  via  cockpit  controls, 
other  crew  station  controls  or  from  the 
instructor  operator  station;  (2)  per¬ 
forms  a  real-time  solution  of  the  sim¬ 
ulator  mathematical  model;  and  (3)  pro¬ 
vides  outputs  necessary  to  accurately 
represent  the  static  and  dynamic 
behavior  of  the  real  world  system  within 
specified  tolerance  and  performance 
criteria. 

Since  TS  are  a  combination  of  interdepen¬ 
dent  hardware  and  software,  a  joint 
development  effort  is  required.  As  the 
complexity  of  TS  increases,  simulation 
software  continues  to  grow  in  com¬ 
plexity,  size  and  cost.  Software  costs 
can  and  do  exceed  computer  hardware 


costs  in  many  cases.  Therefore,  it  is 
imperative  that  the  simulation  software 
acquisition  engineer  process  be  sub¬ 
jected  to  formal  system  engineering  plan¬ 
ning  and  discipline  to  ensure  effective 
and  efficient  simulator  procurement. 

1.3.2  ATE  System  Characteristics 

ATE  consists  of  electronic  devices  cap¬ 
able  of  automatically  or  semi -automati¬ 
cally  generating  and  independently 
furnishing  programmed  stimuli,  measuring 
selected  parameters  of  an  item  being 
tested  and  making  a  comparison  to  accept 
or  reject  the  measured  values  in  accord¬ 
ance  with  predetermined  limits.  ATE  is 
used  in  place  of  manual  devices  either 
because  it  is  more  cost  effective  or  the 
item  being  tested  requires  the  speed  and 
timing  which  only  an  automatic  tester 
can  achieve. 

Figure  1.3-2  shows  the  typical  com¬ 
ponents  of  an  ATE  system.  Note  that 
there  are  both  hardware  and  software  ele¬ 
ments  involved.  Most  of  the  elements 
shown  will  be  found  in'  one  form  or 
another  in  the  majority  of  ATE  systems. 

The  controls  and  displays  section  con¬ 
sists  of  the  computer  and  peripheral 
devices  such  as  control  panels,  magnetic 
tape  cassettes  or  disks,  a  Cathode  Ray 
Tube  (CRT)  and  keyboard,  and  usually  a 
small  printer.  The  computer,  as  con¬ 
trolled  by  software,  performs  tasks  like 
operating  the  peripheral  devices,  switch¬ 
ing  test  stimuli  on  and  off,  and  measur¬ 
ing  and  comparing  responses  of  the  unit 
under  test  (UUT)  to  predetermined 
values.  The  operator  will  maintain  ulti¬ 
mate  control  of  the  testing  process 
through  some  of  the  peripherals.  How¬ 
ever,  his  interaction  is  usually  minimal 
since,  by  definition,  the  automatic  test 
feature  was  selected  in  preference  to  an 
operator -control led  test  system. 

ATE  is  normally  designed  to  allow  a 
single  configuration  of  ATE  to  be  used 
for  testing  several  articles  of  system 
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Figure  1.3-2.  Typical  ATE  Configuration 


equipment.  The  maintenance  level  being 
supported  by  the  ATE  is  determined  by 
logistic  planning. 

The  importance  of  the  software  portion 
of  the  ATE  system  should  not  be  mini¬ 
mized  since  both  the  application  of  the 
test  stimuli  and  the  measurement  of  the 
result  are  achieved  via  software.  In 
some  cases  (not  always)  arbitrary  func¬ 
tion  generation  and  complicated  wave 
analysis  is  also  accomplished  by  soft¬ 
ware. 

1.4  GUIDEBOOK  ORGANIZATION  AND  USE 

The  acquisition  of  computer  programs  for 
ATE  and  TS  systems  has  many  features 
that  are  common  to  the  acquisition  of 
other  types  of  software  and  some 
features  that  are  unique  to  these  sys¬ 
tems.  This  guidebook  focuses  on  these 
unique  features  and  the  problem  areas 
peculiar  to  ATE  and  TS  systems.  The  more 
general  features  common  to  other  types 
of  software  acquisition  are  included, 
but  are  not  given  the  same  emphasis  as 
those  unique  to  ATE  and  TS  systems.  A 
more  general  description  of  computer 
program  documentation  is  given  in  the 
Electronic  Systems  Division  (ESD)  guide¬ 
book  ESD-TR-76-159,  Air  Force  Guide  to 
Software  Documentation  Requirements. 

The  basic  foundation  for  this  guidebook 
is  the  treatment  of  TS  and  ATE  software 
as  configuration  items.  DODD  5000.29, 
Management  of  Computer  Resources  for 
Major  Defense  Systems,  is  explicit  in 
specifying  that  computer  programs  are  to 
be  acquired  as  computer  program  con¬ 
figuration  items  (CPCI).  The  implication 
is  that  computer  programs  will  be 
developed  under  a  separate  account¬ 
ability  and  control.  Computer  programs 
will  be  developed  to  satisfy  a  set  of 
written  requirements  that  have  been 
approved  by  the  Air  Force.  The  develop¬ 
ment  will  include  specific  documentation 
prepared  during  each  development  phase 
which  is  subject  to  the  review  of  the 
project  office.  Development  of  software 


as  a  CPCI  imposes  separate  configuration 
management  controls  and  procedures  on 
the  acquisition  process.  It  provides  pro¬ 
ject  management  with  visibility  and  con¬ 
trol.  It  imposes  disciplines  and  con¬ 
trols  similar  to  hardware  acquisitions. 
In  short,  it  is  a  step  to  move  software 
development  from  an  art  to  an  engineer¬ 
ing  process  with  all  attendant  disci¬ 
plines  and  controls. 

1.4.1  Guidebook  Use 

The  Computer  Program  Documentation 
Requirements  Guidebook  is  designed  to  be 
used  jointly  by  ATE  and  TS  acquisition 
engineers.  This  guidebook  provides  a 
description  of  the  software  acquisition 
engineering  process  as  it  relates  to 
documentation.  For  the  purpose  of  this 
guidebook,  this  process  is  defined  as 
follows: 

a.  Organic  preparation  of  documents. 

b.  Selection  of  a  Contract  Data 
Requirements  List  (CDRL). 

c.  Review  and  approval  of  requested 
documents  produced  by  a  contrac¬ 
tor. 

d.  Document  usage. 

These  are  the  activiites  performed  by 
Air  Force  software  acquisition  engineers 
relating  to  documentation  in  the  soft¬ 
ware  acquisition  process.  The  descrip¬ 
tion  of  these  subjects  is  the  nucleus  of 
the  guidebook.  A  separate  section  is 
devoted  to  each  of  the  four  topics. 

Since  the  acquisition  process  differs 
for  ATE  and  TS,  some  of  the  topics  are 
partitioned  into  two  sections,  one  for 
ATE  and  one  for  TS.  This  separation 
occurs  in  Sections  3.0,  4.0  and  5.0.  It 
is  Intended  that  those  interested  in  TS 
acquistion  read  the  TS  section  and  those 
Interested  in  ATE  acquisition  read  the 
ATE  section.  There  are  Intentional 
redundancies  within  the  two  sections  to 
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make  each  section  complete  and  to 
simplify  the  use  of  the  document  accord¬ 
ing  to  the  user's  particular  needs.  The 
other  sections  of  the  guidebook  are  not 
separated  because  the  subject  matter  Is 
applicable  to  both  ATE  and  TS  disci¬ 
plines. 

1.4.2  Guidebook  Organization 

Section  1.0  of  this  guidebook  contains 
introductory  material  about  the  guide¬ 
book  and  its  relation  to  other  guide¬ 
books.  It  provides  a  brief  description 
of  typical  ATE  and  TS  systems  and 
describes  the  organization  and  use  of 
the  guidebook.  Section  2.0  is  a  list  of 
key  government  documents  that  were  refer¬ 
enced  In  the  preparation  of  this  guide¬ 
book. 

Sections  3  through  7  contain  documenta¬ 
tion  guidelines.  Section  3  provides  a 
discussion  of  the  need  for  documentation 
and  a  summary  of  computer  program  docu¬ 
mentation.  The  sequence  of  documents 
generated  for  ATE  and  TS  software  are 
described  from  the  origin  of  a  weapon 
system  concept  to  the  documents  speci¬ 
fically  produced  for  computer  programs. 


Separate  descriptions  are  provided  for 
ATE  and  TS.  Section  4.0  Is  a  description 
of  documents  that  are  prepared  by  the 
government  In  preparation  for  a  Request 
for  Proposal  (RFP).  Separate  sections 
are  provided  for  ATE  and  TS.  Section  5.0 
Is  devoted  to  the  selection  of  a  CDRL 
for  ATE  and  TS  software  and  In  essence 
Is  a  recommendation  for  a  CDRL  for  each. 
A  checklist  for  the  selection  process  Is 
Included.  Again,  separate  sections  are 
provided  for  TS  and  ATE  computer  pro¬ 
gress.  Section  6.0  describes  the  docu¬ 
mentation  provided  In  each  computer  pro¬ 
gram  development  phase  and  how  the 
development  status  Is  related  to  docu¬ 
mentation.  The  section  also  addresses 
the  review  and  approval  of  the  required 
documents  by  the  Air  Force,  including  a 
review  checklist,  and  the  document  revi¬ 
sion  process.  The  documents  described  In 
Section  4.0  and  5.0  are  grouped  Into  use 
categories  in  Section  7.0  and  are 
related  to  the  purposes  of  the  documenta¬ 
tion.  Section  8.0  Is  a  bibliography  of 
applicable  material.  Section  9  contains 
a  cross  reference  between  guidebook 
topics  and  government  documents.  Section 
10.0,  11.0,  and  12.0  provide  a  glossary 
of  terms,  list  of  abbreviations  and  a 
guidebook  Index,  respectively. 
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Section  2.0  APPLICABLE  DOCUMENTS 


This  Is  a  list  of  key  documents  that  are 
referenced  In  the  text  of  the  guidebook. 
The  referenced  documents  contain 
material  used  In  the  preparation  of  the 
guidebook  and  also  contain  more  detailed 
and  far  reaching  Information  about  the 
subject  than  can  be  Included  In  this 
guidebook.  A  list  of  the  Data  Item 
Descriptions  (DID)  referenced  in  the 
text  is  also  provided. 

2.1  GOVERNMENT  DOCUMENTS 

AFR  310-1,  Management  of  Contractor 
Data,  June  1969 

AFR  800-14,  Vol  II,  Acquisition  and 
Support  Procedures  for  Computer 
Resources  in  Systems,  Sept.  1975 

DOD  5000. 19. L,  Acquisition  Management 
System  and  Data  Requirements  Control 
List,  Jan.  1977 

DODD  5000.29,  Management  of  Computer 
Resources  for  Major  Defense  Systems, 
Apr.  1976 

MIL-D-83468,  Military  Specification  - 
Digital  Computing  System  for  Real-Time 
Training  Simulators,  Dec.  1975 

MIL-STD-483,  Configuration  Management 
Practices  for  Systems,  Equipment,  Muni¬ 
tions  and  Computer  Programs,  Dec.  1970 

MIL-STD-1519,  Preparation  of  Test 
Requirements  Document,  Sept.  1971 

2.2  REFERENCED  DIDs 

DI-A-9324,  Data  Accession  List 
DI-A-5239,  Computer  Program  Development 
Plan 


DI-A-3108,  Conflgurmation  Management 
Plan 

DI-I-3119A,  Computer  Program  Development 
Specification 

DI-E-3120,  Computer  Program  Product 
Specification 

DI-E-3120A/MI ,  Computer  Program  Product 
Specification 

DI-E-3121,  Version  Description  Document 
DI-E-3122,  Configuration  Index 
DI-E-3123,  Change  Status  List 
DI-E-3124,  Specification  Change  Notice 
DI-E-3277,  Training  Equipment  Computer 
Program  Documentation 
DI-H-3277/M3,  Training  Equipment  Com¬ 
puter  Program  Documenta¬ 
tion 

DI-M-5118,  Computer  Software  Maintenance 
Manual 

DI-M-3410,  Computer  Program  User's  Guide 
DI-M-3411,  Computer  Programming  Manual 
DI-H-5070,  System  maintenance  Programs 
(Software) 

DI-T-3703,  Category  I  Test  Plan/ 
Procedures  (Computer  Pro¬ 
grams) 

DI-T-3717,  Category  I  Test  Report  (Com¬ 
puter  Program) 

DI-T-3734,  Test  Requirements  Document 
UDI-E-695/ESD,  Computer  Program  Develop¬ 
ment  Plan 

UDI-S-3911/ASD,  Computer  Program  Develop¬ 
ment  Plan 

DI-E-129/M*,  Computer  Software/Computer 
Program/Computer  Data  Base 
Configuration  Item(s) 

*  Not  a  document  but  the  deliverable 
computer  program  media 


Section  3.0  DOCUMENTATION  SUMMARY 


This  section  provides  a  summary  of  the 
documentation  required  in  the  develop¬ 
ment  of  ATE  and  TS  computer  programs. 
The  need  for  good  documentation  is 
defined.  These  needs  cannot  be  ignored 
particularly  for  TS  and  ATE  computer 
programs  because  of  long  term  mainte¬ 
nance  requirements  and  the  changing 
nature  of  the  systems  being  simulated  or 
tested. 

The  sequence  of  documents  for  both  ATE 
and  TS  systems  are  described  beginning 
with  the  original  weapon  system  ROC  and 
finishing  with  the  documents  prepared  by 
a  contractor  tQ  support  computer  program 
development,  operation  and  maintenance. 
All  documents  related  to  wapon  systems 
acquisition  are  not  discussed,  only 
those  which  are  diredtly  applicable  or 
lead  to  significant  documents  in  com¬ 
puter  program  development  are  described 
herein.  X, 

3.1  DOCUMENTATION  NEEDS 

Documentation  for  system  acquisition  pn£ 
grams  serves  many  needs.  These  needs 
vary  with  the  phase  of  the  computer  pro¬ 
gram  development  cycle.  Each  phase  of 
the  development  cycle  has  unique  charac¬ 
teristics  that  need  to  be  documented. 
Figure  3.1-1  shows  the  type  of  data  that 
is  generated  in  each  phase  of  the  com¬ 
puter  program  development  cycle.  The 
development  in  each  succeeding  phase 
builds  on  data  generated  in  an  earlier 
phase;  thus  demonstrating  the  need  for 
documenting  this  data  as  It  is 
generated.  The  computer  program  develop¬ 
ment  cycle  is  a  repetitive  process  as 
shown  in  the  figure.  Changes,  due  to  new 
requirements  or  design  errors  that  occur 
in  the  operation  and  support  phase, 
initate  the  beginning  of  another  com¬ 
plete  cycle  culminating  in  the  installa¬ 
tion  and  operation  of  the  revised  com¬ 
puter  programs. 

Prior  to  the  computer  program  develop¬ 
ment  phases,  system  concepts,  system 


requirements,  resource  allocation  and 
plans  are  formulated  leading  to  a  bid 
package.  Documentation  of  these  efforts 
Is  essential  since  the  quality  and  often 
the  cost  of  the  delivered  computer  pro¬ 
grams  is  dependent  on  the  quality  of  the 
bid  package.  Errors  introduced  at  this 
stage  are  difficult  and  costly  to  remove 
later. 

Documentation  prepared  in  the  anlaysis 
phase  supports  the  definition  of  the 
functional  performance  requirements  for 
a  computer  program.  Documentation  in 
this  phase  must  address  the  allocation 
of  requirements  from  system  level  speci¬ 
fications,  the  identification  and 
description  of  interfaces  and  the 
analysis  of  alternate  design  approaches. 

Documentation  prepared  in  the  design 
phase  supports  the  development  of  the 
selected  design  approach,  defining  inter¬ 
nal  orogram  structure  and  relationships 
and  completes  the  detailed  program 
descriptions.  During  the  coding  and 
checkout  phase,  the  detailed  design 
description  is  updated  as  necessary  and 
test  procedures  are  finalized.  Computer 
program  code  is  produced  according  to 
the  detailed  design  and  documented  as 
the  program  listing.  The  individual  com¬ 
puter  program  elements  or  modules  are 
tested  against  the  requirements  of  the 
development  specifications  and  the 
design  description  In  the  preliminary 
product  specification  and  are  Integrated 
Into  the  complete  CPCI  in  the  test  and 
Integration  phase.  This  process  involves 
previously  developed  test  plans  and  pro¬ 
cedures  and  results  In  the  generation  of 
reports  on  the  relative  success  of  each 
test  performed. 

The  installation  phase  requires  documen¬ 
tation  to  support  operation  and  mainte¬ 
nance  of  the  software  at  the  operational 
sites.  Since  operational  sites  may  vary 
as  to  the  particular  configuration  or 
operational  requirements,  tests  must  be 
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performed  at  each  site.  This  requires 
additional  test  procedures  and  reports. 

In  the  operation  and  support  phase, 
emphasis  shifts  to  supporting  the  opera¬ 
tional  user  via  manuals  and  handbooks. 
The  support  aspects  require  documenta¬ 
tion  of  the  delivered  software  to  a 
level  that  enables  efficient  correction 
of  deficiencies,  changes  to  existing 
software  and  the  Incorporation  of  new 
requirements.  Configuration  management 
documentation  Is  of  particular  Impor¬ 
tance  during  this  phase  to  ensure  the 
exact  computer  program  configuration  Is 
known.  New  requirements  for  changes  to 
computer  programs  are  documented  as  the 
basis  for  the  beginning  of  a  new  devel¬ 
opment  cycle  to  Incorporate  the  changes. 

As  shown  above,  each  software  develop¬ 
ment  phase  produces  some  unique  docu¬ 
mentation  and  in  turn  is  dependent  on 
documentation  developed  during  some 
previous  phase.  The  following  describe 
the  specific  needs  for  computer  program 
documentation. 

a.  Provide  Planning  and  Allocation  of 
Computer  Resources  throughout  the  Life 
Cycle.  Planning  documentation  is  pro¬ 
vided  In  the  very  early  stages  of  system 
development  and  Is  the  responsibility  of 
the  Air  Force  Commands;  l.e. ,  using. 
Implementing  and  support  commands,  that 
are  Involved  with  the  development,  use 
and  maintenance  of  the  computer  pro¬ 
grams. 

b.  Provide  a  Baseline  to  Establish 
the  Precise  Definition  of  a  Computer  Pro¬ 
gram.  The  development  specification  pro¬ 
vides  an  agreement  between  the  Air  Force 
and  the  contractor  as  to  what  a  computer 
program  must  do,  how  well  It  must  per¬ 
form  and  the  conditions  under  which  It 
must  perform.  The  product  specification 
provides  an  exact  description  of  the  as 
built  and  delivered  computer  program. 
Computer  programs  are  accepted  or 
rejected  by  the  Air  Force  depending  on 


whether  the  functional  and  performance 
requirements  are  satlslfled  and  whether 
It  has  been  demonstrated  that  the  com¬ 
puter  program  code  Is  accurately 
described  by  the  product  specification. 

c.  Provide  a  Means  for  Tracking  Pro¬ 
gress.  Since  unique  documentation  for 
each  development  phase  exists.  It  may  be 
used  as  one  of  the  methods  to  track  the 
progress  of  computer  program  development 
and  to  provide  Information  to  management 
(both  Air  Force  and  contractor)  for  visi¬ 
bility  and  decision  making. 

d.  Provide  a  Means  for  Achieving 
Higher  Quality  Software  Products.  Good 
documentation  through  the  development 
phases  will  aid  understanding  of  com¬ 
puter  program  requirements,  making  It 
easier  to  achieve  a  satisfactory  design. 
Also  it  will  facilitate  reviews  of  soft¬ 
ware  design,  test  plans  and  test  proce¬ 
dures  and  thus  Increase  the  probability 
of  finding  errors  In  the  computer  pro¬ 
gram  design  in  all  stages  of  develop¬ 
ment. 

e.  Provide  an  Orderly  and  Systematic 
Means  of  Communication  at  the  Technical 
Level.  When  design.  Interface  or  test 
data  are  formally  documented,  they  are 
available  to  all  programmers  who  need 
that  data,  are  easy  to  locate  and  pro¬ 
vide  a  link  between  the  programmers 
assigned  to  different  parts  of  the  pro¬ 
gram.  A  design  that  Is  not  documented  Is 
no  design  at  all,  only  a  collection  of 
one  person's  Ideas  that  only  he  can  use 
and  are  many  times  forgotten  and  lost. 

f.  Provide  a  Means  for  Supporting  the 
Operation  and  Maintenance  of  a  Computer 
Program.  Good  documentation  Is  abso¬ 
lutely  essential  for  efficient  operation 
and  maintenance  of  computer  programs. 
Documentation  generated  In  the  develop¬ 
ment  phases  Is  the  only  logical  source 
for  these  data.  Documentation  developed 
after  the  fact  for  operation  and  mainte¬ 
nance  Is  often  very  expensive,  and  more 


often  than  not.  Is  both  Incomplete  (lack¬ 
ing  background  data)  and  Inaccurate. 

3.2  TS  COMPUTER  PROGRAM  DOCUMENTATION 
SUMMARY 

The  purpose  of  this  section  Is  to 
summarize  the  entire  documentation 
sequence  for  TS  computer  programs  and  to 
show  the  relationship  to  the  weapon  sys¬ 
tems  acquisition  life  cycle  and  the  TS 
computer  program  development  cycle. 
Identification  of  the  documents  and  the 
relative  sequence  in  which  they  are  pro¬ 
duced  are  discussed  in  this  section. 
Description  of  the  documents  are  found 
in  paragraphs  4.1  and  5.5.1. 

TS  computer  programs  have  been  acquired 
under  several  different  types  of  con¬ 
tracts.  The  most  common  has  been  a 
separate  procurement  and  a  fixed  price 
contract  in  which  the  entire  TS  system 
is  acquired  as  a  single  configuration 
item.  However,  depending  on  the  com¬ 
plexity  of  the  systems,  some  of  the 
current  TS  acquisitions  consist  of 
multiple  configuration  items  including 
computer  program  configuration  items. 
Figures  3.2-1  and  3.2-2  illustrate  the 
documentation  sequence  for  the  separate 
procurement-fixed  price  method.  The 
figures  show  the  documents  produced 
during  the  respective  phases.  Documenta¬ 
tion  sequences  for  other  types  of  pro¬ 
curement  would  be  similar  and  would 
probably  need  some  slight  modification. 
The  basic  documents  are  essentially  the 
same,  but  some  would  require  a  different 
emphasis. 

Figure  3.2-1  shows  the  applicable  weapon 
system  documentation  sequence  up  to  the 
development  of  a  TS  system.  The  valida¬ 
tion,  full-scale  development  and  Produc¬ 
tion  phases  in  the  weapon  system  life 
cycle  are  started  only  when  approved  by 
the  Defense  System  Acquisition  Review 
Council  (DSARC).  Separate  contracts  are 
usually  awarded  for  each  of  these  three 
phases.  The  TS  computer  program  docu¬ 
mentation  sequence  is  shown  as  a  bar  in 


the  figure.  This  1$  expanded  in  Figure 
3.2-2.  It  is  significant  to  note  the 
time  delay  between  serious  TS  activity 
and  the  beginning  of  a  weapon  system 
acquisition.  The  replacement  of  the  TS 
development  bar  in  Figure  3.2-1  repre¬ 
sents  a  current  weapon  system  pro¬ 
curement  and  the  current  emphasis  on 
crew  training  by  TS  systems.  Some 
current  TS  contracts  such  as  the  B52  and 
KC135  TS  systems  were  started  well  into 
the  deployment  phase  and  trail  the 
acquisition  of  these  weapon  systems  by 
many  years. 

Figure  3.2-2  illustrates  the  development 
phase  relationship  and  the  sequence  in 
which  specific  documents  are  produced 
either  directly  for  or  in  support  of  com¬ 
puter  programs.  Document  sequence  is 
shown  only  in  a  left-right  sequence 
accompanied  by  arrowheads;  vertical  rela¬ 
tionships  are  not  indicative  of 
sequence. 

The  origin  of  all  weapon  systems  devel¬ 
opment  projects  is  the  ROC  or  an  equiva¬ 
lent  document.  The  weapon  system  ROC  may 
call  for  a  training  capability  in  very 
general  terms;  thus,  giving  rise  to  the 
TS  ROC  as  shown  by  the  arrow  relation¬ 
ship  beween  the  weapon  system  ROC  and 
the  TS  ROC  in  Figures  3.2-1  and  3.2-2. 
As  the  weapon  system  development  con¬ 
tinues,  weapon  system  characteristics 
are  defined  and  become  the  basis  for  the 
performance  characteristics  that  will  be 
simulated  by  the  TS  system;  thus,  the 
arrow  relationship  between  the  weapon 
system  characteristics  data  and  the  TS 
documentation  bar  in  Figure  3.2-1. 

Acquisition  of  the  TS  normally  origi¬ 
nates  with  a  TS  system  ROC  or  its  equiva¬ 
lent.  Following  validation  of  the  ROC  a 
Program  Management  Directive  (PMD)  is 
published  which  authorizes  the  devel¬ 
opment  of  the  TS  system.  Project  plan¬ 
ning  for  computer  resources  is  docu¬ 
mented  in  the  Program  Management  Plan 
(PMP)  and  the  Computer  Resource  Inte¬ 
grated  Support  Plan  (CRISP).  Documenta- 
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tlon  for  the  Request  for  Proposal  (RFP) 
package  Is  prepared  and  sent  to  prospec¬ 
tive  contractors.  The  RFP  package 
Includes  a  TS  System  Specification,  Con¬ 
tract  Data  Requirements  List  (CDRL), 
Statement  of  Work  (SOW),  and  the  Informa¬ 
tion  for  Proposal  Preparation  (IFPP). 
Standards  for  Proposal  Evaluation  (SFE) 
are  also  prepared  for  government  use. 

Contractors  prepare  a  proposal  package 
which  includes  the  technical  proposal,  a 
Computer  Program  Development  Plan  (CPDP) 
and  a  Configuration  Mangement  Plan 
(CMP).  The  technical  proposal  and  the 
CPDP  may  be  Included  as  part  of  the  con¬ 
tract.  Up  to  this  point  there  is  no 
distinction  between  TS  software  and  hard¬ 
ware  if  the  TS  system  is  acquired  as  a 
single  configuration  item. 

The  contractor  will  prepare  some  documen¬ 
tation  unique  to  TS  computer  programs, 
some  unique  to  TS  hardware  and  some  that 
covers  the  entire  TS  system.  Documents 
unique  to  computer  programs  are  the  Com¬ 
puter  Program  Product  Specification,  the 
Training  Equipment  Computer  Program  Docu¬ 
mentation  (TECPD)  and  the  Version 
Description  Document  (VDD).  Test  plans/ 
procedures,  test  reports  and  the  docu¬ 
ments  related  to  configuration  manage¬ 
ment  are  written  for  the  TS  system.  If 
computer  programs  are  acquired  as 
separate  CPCIs,  separate  test  plans/ 
procedures,  and  test  reports  would  be 
required  for  computer*  programs. 

Figure  3.2-2  shows  several  documents 
repeated  In  successive  development 
phases.  In  the  case  of  the  design  docu¬ 
ments,  l.e. ,  product  specification  and 
Interface  design  description,  prelimi¬ 
nary  release  are  made  and  updated  as 
design  changes  occur  In  successive 
phases;  configuration  management  docu¬ 
ments,  l.e.,  VDD,  configuration  Index, 
change  status  list  and  Specification 
Change  Notice  (SCN),  are  Initiated  at 
their  first  occurrence  and  updated  con¬ 
tinually  for  the  remainder  of  the  system 
life  cycle. 


3.3  ATE  COMPUTER  PROGRAM  DOCUMENTATION 
SUMMARY 

The  purpose  of  this  section  Is  to 
summarize  the  entire  documentation 
sequence  for  ATE  computer  programs  and 
to  show  the  relationship  to  the  weapon 
systems  acquisition  life  cycle  and  the 
ATE  computer  program  development  cycle. 
Identification  of  the  documents  and  the 
relative  sequence  In  which  they  are 
produced  are  discussed  In  this  section. 
Descriptions  of  the  documents  are  found 
In  paragraphs  4.2  and  5.5.2. 

Computer  programs  for  ATE  have  been 
acquired  under  several  different  types 
of  contracts.  A  common  method  is  to 
supplement  a  weapon  systems  contract  by 
a  Contract  Change  Proposal  (CCP).  The 
contract  change  provides  for  both  ATE 
software  and  hardware.  Alternate  procure¬ 
ment  methods  have  been  to  include  ATE  in 
the  original  prime  contract,  by  Engineer¬ 
ing  Change  Proposal  (ECP),  or  to  award  a 
separate  contract.  Including  ATE  In  the 
original  contract  presents  special  prob¬ 
lems  because  at  the  time  the  contract  is 
awarded  the  extent  to  which  ATE  will  be 
used  Is  not  fully  known,  thus  intro¬ 
ducing  an  additional  risk  factor.  The 
separate  contract  Implies  that  a  weapon 
system  has  already  been  developed  and  Is 
in  the  deployment  phase.  In  such  cases, 
a  separate  contract  would  probably  be 
the  best  approach.  The  separate  contract 
would  entail  a  process  similar  to  the  TS 
RFP  preparation  and  contract  response 
described  In  paragraph  3.2,  with  the 
probable  exception  that  the  contractor 
would  provide  the  development  specifica¬ 
tions.  After  contract  award  the  process 
would  be  similar  to  the  one  described  In 
this  paragraph.  This  paragraph  describes 
the  contract  change  approach.  Documenta¬ 
tion  for  the  other  contract  types  Is 
essentially  the  same,  but  may  require 
some  slight  modification. 

Figures  3.3-1  and  3.3-2  illustrate  the 
document  sequence  for  acquisition  by  con¬ 
tract  change.  The  figures  show  the  docu- 
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merits  produced  during  each  of  the  respec¬ 
tive  phases.  Document  sequence  Is  shown 
only  In  left-right  relationship;  verti¬ 
cal  relationships  are  not  indicative  of 
sequence. 

Figure  3.3-1  shows  the  applicable  weapon 
system  documentation  sequence  up  to  the 
development  of  the  ATE  computer  pro¬ 
grams.  The  validation,  full-scale  devel¬ 
opment  and  production  phases  of  the 
weapon  systems  life  cycle  are  started 
only  when  approved  by  the  DSARC. 
Separate  contracts  are  usually  awarded 
for  each  of  these  three  phases.  The  ATE 
computer  program  documentation  sequence 
is  shown  as  a  bar  in  the  figure.  This  is 
expanded  in  Figure  3.3-2.  It  is  signifi¬ 
cant  to  note  that  there  may  be  a  signifi¬ 
cant  time  delay  between  the  beginning  of 
the  weapon  system  acquisition  process 
and  the  ATE  acquisition  activity.  The 
replacement  of  the  ATE  development  bar 
in  Figure  3.3-1  reflects  a  current 
weapon  system  procurement  in  which  ATE 
activity  was  started  late  in  the  develop¬ 
ment  phase  which  is  several  years  after 
award  of  the  prime  contract.  For  any 
given  system  the  bar  may  move  either 
right  or  left.  Indeed,  some  current 
weapon  system  programs  are  considering 
ATE  requirements  prior  to  issuing  an 
RFP.  This  approach  has  merit  because  it 
involves  ATE  planning  early  in  the 
program  but  also  has  the  disadvantage 
that  insufficient  information  may  be 
available  at  this  time  to  completely 
define  ATE  requirements.  As  stated 
earlier,  ATE  has  also  been  acquired 
after  a  weapon  system  has  been  deployed. 
Figure  3.3-2  illustrates  the  ATE  com¬ 
puter  program  development  phase  relation¬ 
ship  and  the  sequence  in  which  the  docu¬ 
ments  are  produced  either  directly  for 
or  in  support  of  computer  programs. 

It  is  very  difficult  to  determine 
detailed  support  equipment  requirements 
if  the  early  planning  is  usually  delayed 
for  a  considerable  time  after  the 
initial  weapon  system  planning.  The 
delay  is  the  source  of  many  problems  in 


planning  for  ATE  computer  programs. 
Figure  3.3-1  shows  the  documents  devel¬ 
oped  for  the  weapon  systems.  The  initial 
formal  documentation  is  a  ROC.  The 
Weapon  System  ROC  normally  specifies  in 
very  general  terms;  e.g. ,  that  support 
equipment  is  required  to  provide  weapon 
systems  maintenance.  After  the  ROC  is 
validated  a  PMD  is  released  authorizing 
further  program  planning  and  competition 
for  funds. 

A  SPO  cadre  is-,  formed  and  a  study  effort 
is  initiated  to  determine  the  various 
means  of  satisfying  the  ROC  and  PMD 
requirements.  A  Development  Concept 
Paper  (DCP)  is  prepared  and  is  sent  to 
the  DSARC  along  with  the  initial  Inte¬ 
grated  Logistics  Support  Plan  (ILSP). 
Based  on  approval  of  the  DCP  and  budget 
authorization,  the  acquisition  process 
is  underway. 

The  CRISP,  the  PMP  and  the  ILSP  are 
early  program  planning  documents  that 
affect  ATE  computer  programs.  These 
documents  lead  to  a  weapon  system  RFP 
that  is  sent  to  prospective  contractors. 
The  RFP  package  is  similar  to  the  one 
described  for  TS  systems  in  paragraph 
3.2.  Particular  significance  in  the  RFP 
package  is  the  SOW  and  the  CDRL. 

The  SOW  may  require  a  CPDP  and  a  CMP  to 
be  submitted  with  the  contractor  pro¬ 
posal.  This  CPDP  is  normally  written  for 
the  weapon  system  operational  and  sup¬ 
port  computer  programs  and  usually  does 
not  specifically  include  computer  pro¬ 
gram  development  and  configuration  man¬ 
agement  techniques  described  in  the 
weapon  system  documents  that  are  appli¬ 
cable  to  ATE.  The  CMP  is  important  at 
this  stage  because  configuration  manage¬ 
ment  requirements  must  be  specified  for 
ATE  computer  programs,  even  at  this 
early  time. 

Serious  planning  for  ATE  begins  after 
the  weapon  systems  contract  is  awarded 
and  the  contractor  begins  planning  for 
support  equipment.  It  is  noted  that  most 
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of  this  planning  is  a  contractor  task. 
The  Integrated  Support  Plan  (ISP),  Sup¬ 
port  Equipment  Plan,  and  the  Support 
Equipment  Recommendation  Data  (SERD) 
documents  lead  to  the  requirements  for 
ATE  computer  programs  and  the  eventual 
contract  addition  for  ATE.  The  arrowhead 
from  the  SERD  to  the  ATE  development  bar 
indicates  this  relationship.  It  should 
be  noted  that  the  Test  Requirements 
Document  (TRD)  is  usually  prepared  by 
the  developer  of  the  units  to  be  tested 
who  may  be  the  weapons  system  contractor 
or  a  subcontractor.  In  either  case  the 
unit  developer  will  probably  not  be 
directly  associated  with  the  ATE  soft¬ 
ware  organization.  The  TRD  is  the  basis 
for  the  development  specification  for 
ATE  test  software.  This  is  indicated  by 
the  arrowhead  from  the  TRD  to  the  ATE 
development  bar.  Preliminary  TRD  data  is 
also  used  in  the  analyses  that  result  in 
the  SERD. 

Paragraph  3.3-2  defines  the  computer  pro¬ 
gram  development  phase  sequence  of  the 
documents  required  specifically  for  ATE 
computer  program  acquisition.  A  CCP  is 
prepared  to  amend  the  contract.  A 
separate  SOW  is  prepared  to  define  the 
ATE  engineering  tasks  to  be  added  to  the 
prime  contract. 

During  negotiation  for  the  addition  to 
the  contract,  a  CDRL  is  established  or  a 
revision  is  made  to  the  weapon  system 
CDRL.  ATE  computer  programs  are 
separated  into  three  categories:  test 
software,  which  controls  the  actual 
testing  functions;  control  software, 
which  includes  executive,  I/O  drives, 
storage  and  retrieval,  and  data  pro¬ 


cessing  tasks;  and  support  software  that 
includes  compilers,  assemblers,  loaders 
and  system  generation  programs.  All 
three  classifications  are  CPCI. 

Since  the  original  CRISP  covers  a  much 
broader  scope  and  is  prepared  for  in 
advance  of  the  ATE  acquisition,  a 
separate  CRISP  for  ATE  is  preferred. 
Thus,  computer  resource  planning  for  ATE 
is  accomplished  with  ATE  as  the  princi¬ 
ple  subject.  The  CPDP  is  specified  in 
AFR  800-14,  Volume  II,  as  a  requirement 
for  all  CPCIs.  Development  specifica¬ 
tions  for  the  computer  programs  are 
generated  during  the  Analysis  Phase. 
Again  the  TRD  plays  a  key  role  that  is 
discussed  more  thoroughly  in  paragraphs 
5. 5. 2. 1.2  and  5. 5. 2. 1.3. 

Figure  3.3-2  shows  the  progression  of 
the  product  specifications,  interface 
design  documents,  test  plans  and  test 
procedures,  programmers  manual,  user 
guide,  VDD,  configuration  index,  change 
status  list  and  SCN.  The  figure  shows 
several  documents  repeated  in  successive 
development  phases.  In  the  case  of  the 
design  documents  preliminary  releases 
are  made  and  updated  as  design  changes 
occur  in  successive  phases;  test  proce¬ 
dures  are  developed  for  succeeding 
higher  level  tests;  i.e. ,  module  tests, 
CPCI  tests,  ATE  level  tests,  installa¬ 
tion,  etc.;  configuration  management 
documents,  i.e.,  VDD,  configuration 
index,  change  status  list  and  SCN,  are 
initiated  at  the  first  occurrence  and 
updated  as  changes  occur  for  the  life  of 
the  program. 


Section  4.0  GOVERNMENT  PREPARED  DOCUMENTS 


During  early  planning  and  pre-RFP  phases 
in  a  weapon  systems  procurement,  the  Air 
Force  produces  a  number  of  documents 
that  lead  to  the  acquisition  of  TS  and 
ATE  computer  programs.  These  early 
planning  documents  are  primarily  devoted 
to  the  weapon  system  and  in  fact  may 
contain  very  little,  if  any,  specific 
references  to  ATE  and  TS.  This  section 
will  deal  with  the  documents  prepared  by 
the  Air  Force  with  direct  application  to 
the  eventual  acquisition  of  TS  and  ATE. 
TS  and  ATE  are  normally  acquired  by 
different  contracting  methods. 

TS  are  often  acquired  as  a  separate 
fixed  price  contract.  The  development  of 
TS  systems  is  not  always  performed  by 
the  weapon  system  contractor.  A  number 
of  contractors  specialize  in  the  build¬ 
ing  of  personnel  training  systems.  Since 
development  of  a  TS  system  usually  does 
not  occur  until  late  in  the  Full  Scale 
Development  phase,  TS  system  require¬ 
ments  can  be  defined  to  a  satisfactory 
degree  to  permit  fixed  price  contract¬ 
ing.  However,  methods  other  than  fixed 
price  are  also  being  employed.  These 
methods  generally  involve  a  separate  con¬ 
tract  and  are  not  part  of  a  weapon  sys¬ 
tem  contract. 

ATE  is  usually  acquired  by  negotiating  a 
supplemental  agreement  to  a  weapon  sys¬ 
tem  contract.  Other  methods  are  also 
used  such  as  including  ATE  in  the 
initial  weapon  system  contract  or  under 
special  circumstance  awarding  a  separate 
contract.  A  separate  contract  is  usually 
awarded  when  development  has  been  com¬ 
pleted  and  the  weapon  system  already 
deployed.  Usually  ATE  is  provided  by  a 
weapon  system  contractor  because  the 
task  Is  integral  to  the  development  of 
the  components  to  be  tested,  testing 
those  components  in  prototype  system  and 
providing  for  efficient  maintenance  and 
support  In  the  development  phase. 

Since  acquisition  methods  differ  con¬ 
siderably,  this  section  is  divided  into 


separate  sections  for  TS  and  ATE.  While 
essentially  the  same  documents  are  pro¬ 
duced  at  some  stage  in  the  process, 
there  is  a  significant  difference  in 
emphasis  and  tasks  performed  by  ATE  and 
TS  engineering  personnel. 

4.1  TRAINER  SIMULATOR  DOCUMENTS 

The  Air  Force  prepared  documentation  is 
similar  to  that  for  a  weapon  system  pro¬ 
curement  with  a  ROC,  PMP,  CRISP  and  pre¬ 
paration  of  an  RFP.  The  emphasis  is  on 
documentation  for  establishing  a  TS  pro¬ 
gram,  program  planning  and  the  formula¬ 
tion  of  a  bid  package  supported  and 
prepared  by  TS  engineering.  Figure  4.1-1 
shows  the  relationship  of  government  pre¬ 
pared  documents  leading  to  the  RFP  and 
the  eventual  proposal  evaluation. 

Since  TS  systems  are  normally  acquired 
under  a  separate  contract,  the  RFP  pack¬ 
age  will  be  generated  specifically  for 
the  trainer  simulator  system  and  will 
require  significant  activity  from  the  TS 
engineering  support  organizations.  Engi¬ 
neer  support  will  also  be  provided  for 
the  early  program  requirements  and  plan¬ 
ning  documents  such  as  the  ROC,  the  PMD 
and  the  PMP.  The  following  paragraphs 
describe  the  generation  of  these  docu¬ 
ments  including  who  prepares  them,  where 
they  fit  in  the  weapon  system  life  cycle 
and  what  responsibilities  are  fulfilled 
by  the  engineering  support  organization. 

4.1.1  Required  Operational  Capability 

The  ROC  is  a  formal  document  used  to 
Identify  an  operational  need  and  to 
request  a  new  or  Improved  capability  for 
the  operating  forces.  It  1$  prepared  In 
accordance  with  AFR  57-1,  Required  Opera¬ 
tional  Capabilities.  In  the  case  of  a  TS 
system  ROC,  It  specifies  the  need  for  a 
capability  for  training  and  evaluation 
activities  to  support  an  Air  Force 
weapon  system  program.  The  training 
capability  sought  Is  described  In  terms 
of  the  need,  the  capability,  the  opera- 
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tional  environment,  support  and  mainte¬ 
nance  concepts,  concept  of  operations 
and  the  preferred  solution  and  alterna¬ 
tives.  Upon  validation  the  ROC  Is  per¬ 
mitted  to  compete  for  funds  and 
resources  within  the  context  of  total 
Air  Force  requirements,  priorities  and 
objectives. 

At  least  two  ROC's  will  be  published 
that  relate  to  the  TS  systems.  The 
weapon  system  ROC  Is  published  at  the 
outset  of  a  weapon  system  acquisition. 
It  will  contain  at  most  only  a  recogni¬ 
tion  that  a  training  capability  must  be 
provided  to  support  the  weapon  system. 
It  may  or  may  not  specify  any  greater 
detail.  In  either  case  the  reference  Is 
very  general  will  not  require  any  signi¬ 
ficant  engineering  action  other  than  a 
cursory  examination  to  ensure  that  a 
training  capability  Is  Identified. 

Preparation  of  the  TS  ROC  is  normally 
begun  after  development  of  a  weapon  sys¬ 
tem  Is  In  progress.  In  some  cases  the  TS 
ROC  may  be  generated  years  after  the 
weapon  system  has  become  operational 
such  as  the  B-52  and  KC-135  systems.  In 
current  systems  It  is  desired  that  the 
training  capability  will  be  available  at 
the  time  the  weapon  system  becomes  opera¬ 
tional.  To  place  the  TS  ROC  in  the  per¬ 
spective  of  the  weapon  systems'  life 
cycle,  at  least  for  current  weapon  sys¬ 
tems,  the  ROC  is  Initiated  during  either 
the  later  full-scale  development  phase 
or  the  production  phase;  In  either  case 
It  is  after  a  decision  to  enter  the  pro¬ 
duction  phase  (see  Figure  3.2-1).  The  TS 
ROC  Is  Initiated  by  the  using  command 
when  the  need  Is  identified.  When 
requested  by  the  originator,  simulator 
engineering  personnel  assist  In  generat¬ 
ing  the  ROC.  Engineering  should  be  pre¬ 
pared  to  assist  In  determining  budgetary 
cost  Information  and  proposing  alterna¬ 
tive  solutions,  for  satisfying  the  ROC. 
Three  alternatives  are  desired  keyed  to 
the  concept  of  minimal,  operational  and 
expanded  performance. 

a.  Minimum-Essential  Performance  Char¬ 
acteristics, 


b.  Optimum  Performance  Characteris¬ 
tics  (or  preferred),  and 

c.  Expanded  Performance  Charac¬ 
teristics  (expanded  characteristics  for 
enhancement  of  effectiveness,  efficiency 
and  utility.) 

Prior  to  publication  of  the  ROC  the  orig¬ 
inator  will  forward  draft  copies  to  com¬ 
mands  having  mission  responsibility.  Air 
Force  Logistics  Command  (AFLC)  and  Air 
Force  Systems  Command  (AFSC)  will  review 
the  drafts.  Engineering  will  participate 
In  the  review,  providing  comments  and 
recommendations  as  necessary. 

Whether  engineering  participation 
involves  participation  In  the  prepara¬ 
tion  of  the  ROC  or  In  reviewing  the  pro¬ 
posed  ROC,  emphasis  should  be  on  express¬ 
ing  the  operational  requirements  rather 
than  describing  the  technical  approach. 
It  is  often  easy  to  get  carried  away 
into  the  technical  approach  to  the  prob¬ 
lem  rather  than  digging  out  the  required 
characteristics  of  the  TS  systems.  The 
effort  spent  on  this  effort  early  in  the 
acquisition  process  may  have  a  signifi¬ 
cant  effect  on  the  ease  of  achieving  the 
required  capability  and  on  the  final  pro¬ 
duct. 

4.1.2  Program  Management  Directive 

Following  the  validation  of  a  ROC  by  HQ 
USAF,  the  Air  Staff  publishes  a  PMD 
authorizing  the  development  of  the  TS 
system,  thereby  Initiating  the  TS  pro¬ 
ject.  The  PMD  is  written  for  the  TS  sys¬ 
tem  In  accordance  with  AFR  800-2, 
Acquisition  Management.  The  PMD  docu¬ 
ments  the  validation  of  the  ROC  program 
decision,  significant  tasking  of  major 
commands  and  direction  and  guidance  on 
the  expenditures  of  funds.  It  also 
directs  that  plans  be  prepared  for  man¬ 
aging  computer  resources.  The  major  com¬ 
puter  resource  planning  documents  are 
the  PMD,  the  PMP,  the  CRISP  and  the 
CPDP.  The  PMD  Is  concerned  with  the 
identification  of  computer  resources  and 
the  technical  and  managerial  expertise 
for  managing  the  acquisition  of  TS  sub- 
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systems.  This  Includes  management 
expertise  to  focus  attention  on  TS  com¬ 
puter  program  development  and  Integra¬ 
tion  across  the  total  TS  system. 

Normally  there  Is  little  or  no  engineer¬ 
ing  participation  In  writing  the  PMD. 
However,  there  Is  some  precedent  for 
engineering  participation  In  its  prep¬ 
aration.  The  Air  Staff  may  request 
assistance  ranging  from  full  participa¬ 
tion  to  responding  to  specific  questions 
for  a  PMD  written  for  a  TS  system.  TS 
engineering  should  be  prepared  to  pro¬ 
vide  assistance  upon  request. 

4.1.3  Program  Management  Plan 

The  PMP  Is  written  for  the  entire  TS 
system  based  on  the  policies  of  AFR 
800-2,  Acquisition  Management  and  AFR 
800-14,  Volume  II.  It  includes  complete 
training  for  the  acquisition  management 
of  TS  computer  resources.  It  provides  a 
requirement  for  a  CRISP  to  be  prepared. 
Between  the  PMP  and  the  CRISP,  planning 
for  complete  acquisition  management  and 
technical  support  of  computer  resources 
are  provided  over  the  entire  TS  life 
cycle. 

Preparation  of  the  PMP  is  the  responsi¬ 
bility  of  the  TS  program  manager.  Since 
It  Is  one  of  the  major  computer  resource 
documents,  TS  software  engineering  per¬ 
sonnel  will  participate  In  the  prepara¬ 
tion  of  the  PMP.  TS  engineering  will  pro¬ 
vide  the  technical  expertise  for  plan¬ 
ning  the  acquisition  management  of  com¬ 
puter  resources.  The  parts  of  the  PMP 
that  are  concerned  with  computer 
resources  are  specified  In  AFR  800-14, 
Volume  II,  Acquisition  and  Support  Proce¬ 
dures  for  Computer  Resource  In  Systems. 
Since  the  PMP  Is  binding  on  all  parti¬ 
cipating  organizations.  It  Is  essential 
that  the  PMP  receives  a  meaningful 
review  from  all  affected  organizations; 
e.g. ,  the  using.  Implementing  and  sup¬ 
port  commands,  before  Its  publication  to 
ensure  a  meaningful  document. 


4.1.4  Computer  Resources  Integrated 
Support  Plan 

The  CRISP  Identifies  organizational  rela¬ 
tionships  and  responsibilities  for  the 
management  and  technical  support  of  com¬ 
puter  resources  and  Is  prepared  with  the 
guidelines  specified  In  AFR  800-14, 
Volume  II.  It  functions  during  the  full 
scale  development  phase  to  Identify  com¬ 
puter  resources  necessary  to  support  com¬ 
puter  programs  after  transfer  of  program 
management  responsibility  and  system 
turnover  to  the  using  command.  It  con¬ 
tinues  to  function  after  the  transfer  of 
program  management  responsibility  and 
system  turnover  as  the  basic  agreement 
between  the  supporting  and  using  com¬ 
mands  for  management  and  support  of  com¬ 
puter  resources. 

The  CRISP  is  written  as  a  part  of  and  In 
parallel  with  the  PMP.  The  CRISP  Is  pre¬ 
pared  by  a  Computer  Resources  Working 
Group  (CRWG).  The  CRWG  consists  of  repre¬ 
sentatives  of  the  Implementing,  support¬ 
ing  and  using  commands  to  ensure  that 
necessary  elements  of  the  CRISP  are 
Included  In  transfer  and  turnover  agree¬ 
ments.  The  CRISP  and  its  periodic 
updates  are  the  responsibility  of  the 
program  manager  and  must  be  approved  by 
him.  The  CRISP  Is  developed  during  the 
analysis  phase  of  a  TS  system  acquisi¬ 
tion  (prior  to  the  RFP)  and  remains  a 
viable  document  throughout  the  TS  system 
life  cycle.  The  CRISP  Is  updated  as 
necessary  to  reflect  changes  In  computer 
resource  requirements. 

TS  engineering  personnel  will  be  repre¬ 
sented  on  the  CRWG.  During  the  Initial 
formulation  of  the  CRISP,  It  Is  Impor¬ 
tant  that  all  affected  commands  are 
fully  prepared  to  spend  the  time  and 
effort  required  for  the  early  planning 
for  the  support  and  use  of  the  computer 
resources.  Full  and  active  participation 
by  experienced  personnel  In  the  CRWG  Is 
essential  for  effective  computer 
resource  planning.  The  CRWG  chairman 
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should  demand  the  proper  level  of  sup¬ 
port  from  each  of  the  affected  commands. 

4.1.5  Request  for  Proposal 

The  TS  RFP  is  prepared  by  the  project 
office.  It  is  prepared  for  potential 
contractors,  to  define  system  require¬ 
ments,  including  source  selection 
requirements  for  a  TS  system.  For  large 
dollar  projects  the  RFP  must  be  reviewed 
and  approved  by  a  Department  of  Defense 
(DOD)  committee  similar  to  DSARC  before 
continuing  with  the  TS  project.  This  bid 
package  Is  vital  to  the  quality  of  the 
TS  system  that  is  eventually  delivered 
to  the  Air  Force.  A  high  quality  well- 
prepared  RFP  that  clearly  expresses  the 
intention  of  the  Air  Force  is  the  basis 
for  a  high  quality  TS  system.  The  skills 
of  specialists  in  all  areas  of  the  RFP 
should  be  employed.  The  parts  of  the  RFP 
covered  by  this  guidebook  are: 

a.  Statement  of  Work 

b.  Contract  Data  Requirements  List 

c.  Information  for  Proposal  Prepara¬ 
tion 

d.  Trainer  Simulator  System  Specifi¬ 
cations 

4. 1.5.1  Statement  of  Work.  The  TS  SOW 
Is  developed  In  accordance  with  chapter 
7  of  AFSCP  800-6,  Statement  of  Work  Pre¬ 
paration  Guide.  The  SOW  defines  the 
requirements  of  tasks  to  be  performed  by 
the  contractor  In  the  design,  develop¬ 
ment,  test  and  evaluation  of  the  TS  sys¬ 
tem. 

The  SOW  Is  prepared  by  the  project 
office  In  the  preparation  of  an  RFP  for 
potential  TS  contractors.  A  statement  of 
work  may  be  prepared  to  cover  each  phase 
of  a  major  weapon  system  contract;  l.e. , 
conceptual  validation,  full-scale  devel¬ 
opment  and  production.  The  SOW  for  a  TS 
system  Is  concerned  only  with  a  full- 
scale  development  and  Is  the  part  of  the 


RFP  that  Identifies  the  full-scale  devel¬ 
opment  tasks  of  design,  development, 
test  and  evaluation  of  the  TS  system 
based  on  the  system  specifications  pro¬ 
vided  as  part  of  the  RFP.  The  Intended 
output  Is  a  hardware  and  software  con¬ 
figured  system  and  the  documentation 
needed  for  Inventory  use.  All  tasks  that 
the  contractor  Is  expected  to  perform 
should  be  explicitly  stated.  When  data 
are  expected  from  a  task,  the  task 
description  should  be  sufficiently 
detailed  to  cause  generation  of  the 
desired  Information. 

The  Initial  SOW  Is  prepared  by  the  TS 
project  office  and  is  Included  as  part 
of  the  RFP.  The  final  SOW  will  normally 
be  the  result  of  the  contractor's  expan¬ 
sion  of  the  Initial  SOW  as  developed 
during  the  contract  negotiation. 

The  SOW  is  written  to  concide  with  the 
Work  Breakdown  Structure  (WBS)  which  per¬ 
mits  a  logical  arrangement  of  the  ele¬ 
ments  of  the  SOW  and  a  tracing  of  work 
effort  expended  under  these  elements. 
The  WBS  is  described  In  MIL-STD-881  and 
AFSCM  173-4. 

TS  engineering  support  Is  required  to 
provide  technical  task  descriptions  for 
the  development  of  the  TS  system  and  for 
specific  documentation  required  In 
accomplishment  of  these  tasks.  Configura¬ 
tion  Items  should  be  Identified  In  the 
SOW  If  there  are  more  than  one.  If  com¬ 
puter  programs  are  to  be  developed  as 
CPCIs,  the  SOW  should  make  that  distinc¬ 
tion  with  the  appropriate  Identification 
of  documents  required. 

It  Is  essential  that  data  rights  be 
obtained  for  all  computer  program  data 
that  are  required  for  efficient  opera¬ 
tion  and  maintenance  of  the  computer  pro¬ 
grams  to  be  delivered.  The  SOW  should 
contain  a  reference  to  ASPR  paragraph 
7-104. 9(A)  that  specifies  the  appropri¬ 
ate  data  rights  provisions. 
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4. 1.5. 2  Contract  Data  Requirements 
List.  The  CDRL  is  the  list  of  data 
requirements  that  are  required  to  be 
submitted  to  the  government  by  a  corn 
tractor  as  a  result  of  a  specific  con¬ 
tract.  The  CDRL  constitutes  the  sole 
list  of  contractual  requirements  for  the 
amount  and  kinds  of  data  required  under 
the  contract.  AFR  310-1,  Management  of 
Contractor  Data,  describes  the  CDRL  and 
procedures  for  specifying  data  require¬ 
ments  for  a  contract. 

Selection  of  the  CDRL  is  the  responsi¬ 
bility  of  the  project  office.  The  CDRL 
is  a  list  containing  all  the  data 
required  for  delivery  to  the  Air  Force 
in  the  fulfillment  of  the  contract,  and 
references  the  appropriate  DIDs.  Since 
the  TS  is  often  acquired  as  a  single 
configuration  item  (Cl),  rather  than  a 
hardware  Cl  and  a  CPCI,  much  of  the  docu¬ 
mentation  specified  by  the  CDRL  will 
apply  to  the  entire  system.  Certain  docu¬ 
ments  apply  only  to  software  and  have 
DIDs  that  are  specially  designed  for  TS 
software,  these  documents  are  the  Com¬ 
puter  Program  Product  Specification  and 
the  Training  Equipment  Computer  Program 
Documentation.  The  CDRL  is  of  vital 
importance  to  the  successful  acquisition 
of  a  TS  system  and  its  eventual  support. 
For  each  TS  acquisition,  engineering 
personnel  should  examine  the  CDRL  to 
assure  all  necessary  documentation  is 
provided.  The  CRISP  describes  all  com¬ 
puter  resource  support  requirements  and 
should  be  used  as  a  checklist  for  docu¬ 
mentation  support.  Documentation  should 
provide  for  adequate  product  baseline 
identification,  means  to  verify  that  the 
delivered  software  satisfies  the  TS  sys¬ 
tem  specification  requirements,  mainte¬ 
nance  provisions,  programming  guidelines 
for  the  computer  program  system  being 
used,  and  configuration  management  data. 
CDRL  selection  and  data  descriptions  are 
described  more  fully  in  Section  5. 

4. 1.5.3  TS  System  Specification.  The 
TS  System  Specification  provides  the 
functional,  performance  and  quality 


assurance  requirements  for  a  TS  system. 
TS  systems  are  often  acquired  as  single 
configuration  items,  and  are  not  further 
broken  down  into  a  CPCI  and  a  hardware 
Cl.  There  are  conditions  when  the  com¬ 
plexity  of  a  TS  system  dictates  that 
more  than  one  configuration  Item  be 
developed.  This  may  also  include  com¬ 
puter  programs  as  CPCIs.  When  the 
requirement  for  multiple  configuration 
Items  are  established,  the  system  speci¬ 
fication  will  be  comprised  of  the 
separate  specifications  for  the  configur¬ 
ation  items. 

The  TS  System  Specification  Is  the 
responsibility  of  the  project  office  and 
is  written  by  TS  engineering  personnel. 
The  system  specification  is  a  critical 
milestone  In  the  acquisition  of  a  TS  sys¬ 
tem.  It  Is  the  end  result  of  the  system 
engineering  process  of  analyses  and 
trade-off  studies  conducted  by  engineer¬ 
ing  in  support  of  project  office.  Charac¬ 
teristics  of  the  weapon  system  to  be 
simulated  are  not  included  in  the  text 
of  this  specification,  but  are  refer¬ 
enced  to  the  weapons  system  characteris¬ 
tics  data  provided  by  the  weapon  system 
contractor.  It  forms  the  allocated  base¬ 
line  from  which  a  contractor  will 
design,  test  and  Install  a  TS  system.  It 
is  therefore  incumbent  on  the  engineers 
to  ensure  that  the  requirements  speci¬ 
fied  are  accurate,  clearly  written  and 
are  capable  of  being  verified.  The  TS 
system  specification  is  prepared  using 
MIL-D-83468,  Military  Specification, 
Digital  Computing  Systems  for  Real-Time 
Training  Simulators,  which  covers  the 
general  characteristics  and  configura¬ 
tions  of  digital  computational  systems 
used  In  real-time  TS  and  also  provides 
general  guidelines  for  mathematical 
models.  It  contains  sections  covering 
software  requirements  and  computer  hard¬ 
ware  requirements.  Verification  of  these 
requirements  are  performed  at  the  system 
level  and  not  at  the  hardware/software 
level  unless  computer  programs  are 
designated  as  CIs.  The  guidebook  for 
requirements  specification  provides  addl- 
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tional  detail  on  the  TS  system  specifica¬ 
tion. 

4. 1.5.4  Instructions  for  Proposal  Pre¬ 
paration.  The  IFPP  provides  the  prospec¬ 
tive  contractors  with  a  detailed  descrip¬ 
tion  of  what  Is  expected  In  their  pro¬ 
posals.  As  such,  this  document  Is  of 
particular  Importance.  It  Is  prepared  by 
the  project  office  as  part  of  the  RFP 
package.  It  contains  the  detailed 
Instructions  for  the  preparation  of  the 
proposal.  It  Is  a  key  document  to  obtain¬ 
ing  high  quality  and  consistent 
responses  from  the  bidders.  It  is 
written  to  be  consistent  with  the  evalua¬ 
tion  criteria  specified  in  the  SFE  docu¬ 
ment.  In  short  it  pulls  the  entire  bid 
package  together  and  facilitates  the 
evaluation  process.  The  quality  of  the 
bid  package  is  greatly  affected  by  this 
document. 

The  Air  Force  identifies  content  require¬ 
ments  for  the  bidders'  proposal.  It  pro¬ 
vides  information  as  to  the  delivery 
criteria  such  as  submission  dates, 
number  of  pages  in  the  various  parts  of 
the  proposal  and  the  general  scope  of 
the  contract.  It  may  also  specify  the 
volumes  to  be  submitted  including  number 
of  pages,  number  of  figures,  paragraph 
and  subparagraph  titles  and  instruction 
for  the  contents  of  each. 

Engineering  should  play  a  key  role  in 
the  organization  and  description  of  the 
technical  material  to  be  obtained  from 
the  bidders.  Evaluation  of  bidders'  pro¬ 
posals  is  simplified  by  proper  attention 
to  organization  and  content  of  the  pro¬ 
posal  volumes.  The  use  of  well-qualified 
and  experienced  engineers  in  the  pre¬ 
paring  the  IFPP  is  essential  for 
obtaining  the  desired  response  from  the 
bidders. 

4.1.6  Standards  for  Evaluations 

The  SFE  is  prepared  by  the  project 
office  prior  to  completing  the  RFP.  TS 
engineering  will  provide  support  to  the 


project  office  by  specifying  significant 
technical  criteria  for  evaluating  the 
bidders'  proposals.  These  criteria  are 
used  by  the  proposal  review  teams  to 
ensure  a  consistent  level  of  evaluation 
among  the  reviewers.  The  SFE  should 
Include  a  checklist  of  significant 
points  to  be  considered  with  appropriate 
weighting  factors.  The  SFE  Is  not 
included  in  the  RFP  and  Is  not  Intended 
for  distribution  to  the  bidders.  Rather, 
the  criteria  for  evaluation  should  be 
reflected  in  the  elements  that  make  up 
the  RFP,  primarily  in  the  IFPP  and ‘the 
SOU. 

4.2  AUTOMATIC  TEST  EQUIPMENT 
DOCUMENTS 

ATE  is  normally  acquired  as  part  of  a 
major  weapon  system.  One  of  the  signi¬ 
ficant  problems  in  ATE  acquisition  is 
the  considerable  time  lag  between  the 
initial  planning  documentation  for  the 
weapon  system  and  the  beginning  of 
serious  activity  for  ATE  acquisition. 
Participation  in  the  early  planning 
stages  is  necessary  to  assure  that 
proper  provisions  for  ATE  software  are 
recognized  and  planned  for  during  the 
development  of  the  weapon  system. 

There  is  little  documentation  prepared 
by  the  Air  Force  that  is  directly  appli¬ 
cable  to  the  acquisition  of  ATE  computer 
programs.  The  early  program  requirements 
and  planning  documents  are  directed 
primarily  to  the  weapon  system  which  the 
ATE  will  support.  The  weapon  system  ILSP 
and  the  CRISP  provide  an  opportunity  for 
some  early  planning.  Normally  ATE 
requirements  are  not  included  in  a 
weapon  system  RFP.  ATE  is  usually 
acquired  by  adding  to  the  prime  contract 
via  a  CCP.  However,  there  are  occasions 
when  a  separate  RFP  is  issued  specifi¬ 
cally  for  ATE.  Typically,  the  weapon  sys¬ 
tem  RFP  contains  a  CDRL  that  requires 
coordination  by  ATE  engineering  per¬ 
sonnel.  Figure  4.2-1  categorizes  the 
documents  prepared  by  the  Air  Force  into 
program  requirements,  program  planning 
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and  contract  packages.  In  general,  docu- 
ments  In  these  categories  are  Initially 
prepared  In  the  time  periods  separated 
by  the  respective  DSARC. 

The  most  prominent  documents  prepared 
for  ATE  acquisition  are  the  CRISP,  the 
weapon  system  SOW  Including  the  CDRL  for 
the  weapon  system  contract  and  the  CCP 
for  ATE.  If  ATE  Is  to  be  acquired  under 
a  separate  contract,  the  RFP  becomes  of 
vital  Importance.  The  following  para¬ 
graphs  describe  the  preparation  of  these 
documents,  where  they  fit  In  the  weapon 
system  life  cycle  and  what  responsi¬ 
bilities  are  fulfilled  by  the  ATE  engi¬ 
neering  personnel. 

4.2.1  Required  Operational  Capability 

The  ROC  Is  a  formal  document  used  to 
Identify  an  operational  need  and  to 
request  a  new  or  Improved  capability  for 
operating  forces  and  Is  prepared  In 
accordance  with  AFR  57-1,  Required  Opera¬ 
tional  Capabilities.  There  usually  Is 
not  a  ROC  Issued  specifically  for  sup¬ 
port  equipment.  In  such  cases  the  weapon 
system  ROC  will  call  for  support  equip¬ 
ment  only  In  very  general  terms,  e.g. , 
support  equipment  Is  required  to  provide 
a  maintenance  capability  for  the  weapons 
system. 

The  ROC  applicable  to  the  acquisition  of 
ATE  Is  usually  the  weapon  system  ROC. 
The  generation  of  operational  require¬ 
ments,  consisting  of  statements  of  need 
and  the  operational  capability.  Is  nor¬ 
mally  an  activity  of  a  using  command 
with  the  collaboration  of  the  AFSC,  AFLC 
and  ATC.  Preparation  of  the  weapon  sys¬ 
tem  ROC  Is  the  earliest  stage  In  the 
acquisition  process.  Therefore,  It  is 
necessarily  a  relatively  high-level  docu¬ 
ment  that  describes  the  need  for  a 
weapon  system  and  specifies  the  required 
characteristics  of  the  system.  One  of 
these  characteristics  Is  the  need  for 
support  equipment.  At  this  point  It  Is 
too  early  to  know  If  any  or  how  much  ATE 
will  be  required.  In  all  probability  ATE 


engineering  will  not  be  invited  to  parti¬ 
cipate  In  the  formulation  of  the  weapon 
system  ROC.  The  weapon  system  ROC  should 
be  reviewed  by  ATE  engineering  to  ensure 
provisions  for  support  equipment  are 
Included.  The  Inclusion  of  appropriate 
support  equipment  provisions  will  fore¬ 
stall  future  problems  when  the  need  to 
acquire  ATE  becomes  evident.  Detailed 
requirements  for  ATE  probably  are  not 
available  at  the  time  the  ROC  Is  gener¬ 
ated,  but  useful  basic  guidance  can  fre¬ 
quently  be  suggested  when  problems  are 
foreseen. 

In  some  cases,  when  a  requirement  for 
ATE  develops  for  an  operational  weapon 
system,  an  ATE  ROC  may  be  formulated.  It 
is  probable  that  ATE  engineering  would 
then  be  requested  to  participate  In  pre¬ 
paring  this  ROC.  An  ATE  ROC  would  con¬ 
tain  a  great  deal  more  detail  than  the 
weapon  system  ROC  and  would  greatly 
benefit  from  the  experience  represented 
In  the  ATE  engineering  support  per¬ 
sonnel.  ATE  engineering  should  be  pre¬ 
pared  to  assist  In  determining  budgetary 
cost  estimates  and  alternative  solutions 
for  satisfying  the  ROC.  Three  alterna¬ 
tives  are  desired,  keyed  to  the  concept 
of  minimal,  optional  and  expanded  per¬ 
formance: 

a.  Minimal-— Essential  Performance 
Characteristics 

b.  Optimum  Performance  Characteris¬ 
tics  (or  Preferred) 

c.  Expanded  Performance  Characteris¬ 
tics  (expanded  characteristics  for 
enhancement  of  effectiveness,  efficiency 
and  utility). 

Whether  ATE  engineering  participates  In 
preparation  of  the  ROC  or  In  reviewing 
the  proposed  ROC,  emphasis  should  be 
focussed  on  expressing  operational 
requirements  rather  than  describing  the 
technical  approach  to  a  solution.  It  Is 
easy  to  get  "carried  away"  Into  a  philo¬ 
sophical  technical  approach  rather  than 


digging  out  the  required  characteristics 
of  an  ATE  system. 

4.2.2  Program  Management  Directive 

The  PMD  is  published  after  validation  of 
the  ROC  and  the  ensuing  weapons  system 
trade-off  studies.  The  PMD  authorizes 
the  development  of  the  weapon  system, 
makes  it  eligible  to  compete  for  funds 
and  directs  that  plans  be  prepared  for 
managing  computer  resources.  The  major 
computer  resource  planning  documents  are 
the  CRISP  and  the  PMP.  Preparation  of 
the  PMD  is  described  in  AFR  800-2 
"Acquisition  Management".  It  should  be 
noted  that  the  PMD  is  concerned  with  the 
weapon  system  and  does  not  directly 
address  ATE  software. 

After  receipt  of  the  PMD,  a  study  effort 
is  initiated  by  a  Systems  Program  Office 
(SPO)  cadre  to  determine  various  means 
for  satisfying  the  ROC.  A  Develop  Con¬ 
cept  Paper  (DCP)  is  prepared  containing 
a  record  of  basic  program  information, 
decision  rationale  and  review  thres¬ 
holds.  The  DCP  and  the  initial  ILSP  are 
prepared  for  review  by  DSARC  I.  When 
approved  the  DCP  serves  as  authority  to 
proceed  with  a  particular  phase  of  the 
acquisition  cycle.  The  SPO  cadre  then 
becomes  a  SPO  and  work  is  begun  on  an 
RFP. 

The  PMD  and  the  DCP  are  published  by  HQ 
USAF  and  the  SPO  cadre,  respectively. 
Since  these  documents  are  weapon  system 
oriented  and  are  far  removed  in  time 
from  ATE  activity,  there  is  no  ATE  engi¬ 
neering  participation  in  their  prepara¬ 
tion.  These  documents  are  usually  pub¬ 
lished  in  the  weapon  system  conceptual 
life  cycle  phase.  At  that  time  there 
have  been  no  specific  requirements  devel¬ 
oped  for  ATE,  only  the  realization  that 
for  the  type  of  systems  being  developed 
the  use  of  ATE  is  highly  probable. 


4.2.3  Integrated  Logistics  Support 
Plan 

The  ILSP  is  a  document  which  provides  a 
comprehensive  and  detailed  plan  for 
implementing  the  concepts,  techniques 
and  policies  necessary  to  achieve  the 
integrated  logistics  support  objectives. 
These  are  assuring  the  effective  econo¬ 
mical  support  of  a  logistics  elements 
into  program  planning,  development,  test 
and  evaluation,  production  and  opera¬ 
tional  processes. 

The  ILSP  is  prepared  by  an  Integrated 
Logistics  Support  Office  (ILSO)  for  and 
under  the  guidance  of  the  program 
manager.  It  is  published  initially 
during  the  conceptual  life  cycle  phase 
of  a  weapon  system  for  which  Integrated 
Logistics  Support  (ILS)  is  applicable 
and  specified  in  the  PMD  and  PMP.  Sup¬ 
port  and  test  equipment.  Including  ATE, 
is  one  of  the  ILS  elements  that  are  con¬ 
sidered  in  the  plan.  As  the  system  or 
equipment  life  cycle  progresses,  the 
ILSP  continually  grows  from  Its  initia¬ 
tion  in  the  conceptual  phase  and  evolves 
with  ever-increasing  availability  of 
information.  The  material  contained  in 
the  ILSP  regarding  ATE  will  determine 
the  scope  of  coverage  to  be  contained  in 
the  RFP.  The  ILS  program  is  described  In 
AFR  800-8,  Integrated  Logistics  Support 
Program  For  Systems  and  Equipment. 

ATE  engineering  support  is  net  provided 
for  the  Initial  release  during  the  early 
stages  of  system  development.  As  require¬ 
ments  for  ATE  are  formulated  and  imple¬ 
mented,  the  available  Information  should 
be  made  available  to  the  ILSO  for  Inclu¬ 
sion  in  the  ILSP. 

4.2.4  Request  for  Proposal. 

ATE  and  its  supporting  computer  pro¬ 
grams,  are  normally  acquired  by  an  addi- 


tlon  to  the  primary  weapon  system  con¬ 
tract.  Occasionally  ATE  may  be  procured 
under  a  separate  contract.  The  prepara¬ 
tion  of  RFP  documentation  Is  similar  to 
that  described  for  TS  systems  In  para¬ 
graph  4.1. 

There  Is  often  a  time  span  of  up  to 
several  years  between  the  time  the 
weapon  system  RFP  Is  prepared  and 
Identification  of  the  required  ATE  and 
Its  computer  programs.  In  spite  of  the 
long  time  span,  there  are  some  important 
Items  In  the  weapon  system  RFP  that  need 
the  early  attention  of  ATE  software  engi¬ 
neering.  The  CDRL  and  the  SOW  are  of 
particular  Importance.  The  CDRL  requires 
close  attention  even  though  specific  ATE 
and,  therefore,  the  supporting  computer 
programs  cannot  yet  be  Identified.  What 
ATE  provisions  there  are  In  the  SOW  are 
scattered  throughout  with  no  formal 
level  of  tasking,  making  It  difficult  to 
find  and  relate  all  ATE  areas.  ATE  engi¬ 
neering  participates  in  the  formulation 
of  the  RFP.  Even  though  active  participa¬ 
tion  will  be  minimal,  competent  and 
highly  experienced  personnel  are 
required.  Their  experience  will  deter¬ 
mine  the  effectiveness  of  the  ATE  inputs 
to  the  RFP.  The  assumption  should  be 
made  for  modern  major  weapon  systems 
that  ATE  software  will  be  required. 

The  CDRL  is  of  particular  importance 
because  it  is  the  contractual  means  for 
obtaining  data  required  from  the  contrac¬ 
tor.  The  CDRL  constitutes  the  sole  list 
of  contractual  requirements  for  the 
amounts  and  kinds  of  data  required  under 
a  given  contract.  AFR  310-1,  Management 
of  Contractor  Data,  describes  the  CDRL 
and  procedures  for  specifying  data 
requirements  for  a  contract. 

ATE  engineering  should  provide  a 
recommended  CDRL  for  ATE  software  (see 
paragraphs  5.3  and  5.2.2).  Since  this 
recommendation  is  made  for  the  Imple¬ 
menting,  using  and  support  agencies.  It 
Is  Important  that  all  these  agencies  are 
consulted.  Often,  the  ATE  software  CDRL 


will  be  combined  with  the  operational 
software  CDRL  resulting  In  DIDs  that 
contain  the  same  titles,  but  have  no 
references  to  ATE  software.  The  approved 
CDRL  and  list  of  DIDs  must  be  reviewed 
carefully  to  ensure  that  adequate  pro¬ 
visioning  Is  made  for  ATE  software.  The 
pitfall  in  not  specifying  the  applicabil¬ 
ity  to  ATE,  Is  that  the  contractor  may 
claim  he  is  not  required  to  provide  the 
desired  documentation  since  It  Is  not 
specified  in  the  CDRL  and  the  referenced 
DIDs.  The  ATE  engineer  must  aggressively 
pursue  the  development  and  progress  of 
the  CDRL  to  ensure  references  to  ATE 
documentation  are  not  Inadvertently 
removed  from  the  approved  list  and  that 
ATE  provisions  are  represented  In  the 
approved  DIDs.  If  ATE  is  acquired  by  con¬ 
tract  supplement,  the  CDRL  can  be 
updated  or  a  separate  CDRL  can  be  pre¬ 
pared  for  ATE.  The  CDRL  should  be  exam¬ 
ined  carefully  to  assure  that  provisions 
for  ATE  are  adequately  covered  and  any 
needed  corrections  or  additions  are 
Included. 

ATE  engineering  should  also  review  the 
SOW  and  the  exhibits  to  ensure  that  the 
provisions  for  ATE  that  are  scattered 
throughout  the  RFP  are  accurate  and 
representative  of  the  weapon  system. 
Since  they  are  scattered,  the  entire  RFP 
should  be  examined  carefully.  Adequate 
provisioning  even  at  this  early  date  may 
save  negotiating  time  and  contract  money 
if  and  when  a  contract  supplement  Is 
required.  It  is  even  more  important  If 
ATE  software  is  to  be  acquired  directly 
through  the  prime  contract.  Then  It  must 
be  clearly  stated  that  ATE  is  to  be  pro¬ 
vided  as  part  of  the  support  equipment 
and  a  more  specific  definition  of  ATE 
requirements  must  be  provided.  The  con¬ 
tractor  must  be  given  the  latitude  to 
perform  trade-off  studies  to  determine 
the  best  mix  of  ATE  and  other  support 
equipment  that  will  satisfy  RFP  require¬ 
ments. 

It  Is  essential  that  data  rights  be 
obtained  for  all  computer  program  data 
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that  are  required  for  efficient  opera¬ 
tion  and  maintenance  of  the  computer  pro¬ 
grams  to  be  delivered.  The  SOU  should 
contain  a  reference  to  ASPR  paragraph 
7-104.9(A)  that  specifies  the  appropri¬ 
ate  date  rights  provision. 

ATE  Is  a  high  cost  Item  that  requires 
specialized  equipment  and  personnel.  Nor¬ 
mally  there  are  no  provisions  for  ATE  in 
a  UBS  for  cost  collection.  Since  a  SOW 
is  structured  around  the  UBS,  ATE  provi¬ 
sions  are  scattered  throughout  the  SOW. 
Provisions  should  be  made  at  the  WBS 
level  for  collection  of  ATE  costs  thus 
allowing  ATE  provisions  to  be  localized. 
This  would  not  only  simplify  the  ATE 
inputs  to  the  SOW  and  their  review  but 
would  focus  attention  on  a  significant 
portion  of  the  weapon  system  that  Is 
often  overlooked. 

4.2.5  Contract  Change  Proposal 

After  approval  by  the  government  a  CCP 
provides  a  means  for  adding  support 
equipment  to  the  weapon  systems  cdn*— 
tract.  It  will  authorize  the  acquisition 
of  support  equipment,  including  ATE,  and 
will  provide  the  necessary  modification 
to  the  CDRL  and  to  the  SOW  to  the 
primary  weapons  system  contract.  The  CCP 
should  require  that  a  CPDP  be  prepared 
which  Includes  the  contractor's  plan  for 
developing  the  three  types  of  ATE  soft¬ 
ware,  i.e. ,  support  software,  control 
software  and  test  software. 

The  long  time  span  between  the  weapon 
system  RFP  and  procurement  of  ATE  and 
the  fact  that  ATE  is  a  high  cost  item 
leads  to  the  need  for  a  contract  supple¬ 
ment  for  ATE.  The  desired  configuration 
of  ATE  depends  In  part  on  test  require¬ 
ments  specified  during  the  development 
of  the  UUTs  used  In  the  Optimum  Repair 
Level  Analysis  (ORLA)  and  the  SERB  docu¬ 
ments.  All  of  the  above-mentioned 
analyses  and  documents  are  prepared  by 
the  contractor  and  reviewed  and  approved 
by  the  Air  Force.  The  ATE  engineering 
organization  is  directly  involved  with 


this  process  and  should  keep  abreast  of 
these  analyses,  carefully  review  the 
resulting  documentation  and  provide  com¬ 
ments  and  recommendations  to  support 
final  results. 

The  CCP  Is  normally  prepared  jointly  by 
the  contractor  and  the  Air  Force.  The 
CCP  will  contain  a  separate  SOW  that  Is 
subject  to  negotiations  between  the  con¬ 
tractor  and  the  Project  Office.  ATE  engi¬ 
neers  provide  technical  consultation  and 
direction  In  the  preparation  of  the  CCP. 
This  Includes  agreement  on  the  CDRL,  the 
SOW  and  the  Implementation  schedule. 
Items  that  were  not  contracted  In  the 
prime  contract  must  be  negotiated  for 
the  CCP.  Preparation  of  the  CCP  should 
be  closely  monitored  during  Its  prepara¬ 
tion  to  minimize  changes  In  the  review 
and  approval  by  the  SPO. 

4.2.6  Program  Management  Plan 

The  PMP  Is  concerned  with  the  identifica¬ 
tion  of  computer  resources  and  the  tech¬ 
nical  and  managerial  expertise  for 
nfimiglng  the  acquisition  of  weapon  sys¬ 
tem  software.  It  Is  usually  written  for 
the  weapon  system  and  Is  based  on  AFR 
800-2,  Acquisition '-Management,  supple¬ 
ment  1  and  AFP  800-14,"  Volume  II, 
Acquisition  and  Support  Procedure* .  for 
Computer  Resources  In  Systems.  At  the 
time  the  PMP  Is  originally  released, 
there  Is  no  specific  coverage  of  ATE 
computer  resources.  When  ATE  computer 
resources  are  Identified,  a  change  to 
the  PMP  will  be  published  covering  such 
resources.  The  PMP  provides  a  require¬ 
ment  for  a  CRISP  to  be  prepared. 

t 

The  PMP  and  the  CRISP  provide  complete 
planning  for  acquisition  management  and 
technical  support  of  computer  resources 
Including  ATE  for  the  entire  life  cycle 
of  the  weapon  system. 

The  PMP  Is  prepared  by  the  SPO.  Since  It 
Is  weapon  system  oriented,  there  is  no 
ATE  engineering  participation  In  Its  pre- 
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paration.  The  PMP  is  usually  published 
in  the  weapon  system  validation  life 
cycle  phase. 

In  the  event  a  PMD  and  PMP  are  written 
specifically  for  an  ATE  acquisition,  ATE 
engineering  personnel  would  participate 
in  formulating  the  PMP.  ATE  engineering 
should  be  prepared  to  provide  the  techni¬ 
cal  expertise  for  planning  the  acquisi¬ 
tion  management  of  computer  resources. 
The  PMP  is  binding  on  all  participating 
organizations;  therefore,  it  is 
Important  that  the  ATE  PMP  receive  a 
meaningful  review  from  all  affected 
organizations,  i.e. ,  using,  implementing 
and  support  commands,  before  it  Is  pub¬ 
lished. 

Some  current  programs  are  attempting  to 
provide  an  early  identification  of  ATE 
requirements  and  required  resources. 
This  practice  should  be  encouraged.  The 
earlier  planning  is  begun  the  easier  the 
eventual  acquisition  will  be. 

4.2.7  Computer  Resources  Integrated 
Support  Plan 

The  CRISP  identifies  organizational 
relationships  and  responsibilities  for 
the  management  of  technical  support  of 
computer  resources  as  specified  in  AFR 
800-14,  Volume  II,  Acquisition  and  Sup¬ 
port  Procedures  for  Computer  Resources 
In  Systems.  The  CRISP  functions  during 
the  full-scale  development  phase  to 
identify  computer  resources  necessary  to 
support  computer  programs  after  transfer 
of  program  management  responsibility 
from  the  Implementing  cutisnand  to  the 
using  and  support  commands.  It  coniines 
to  function  after  the  transfer  of  pro¬ 
gram  management  responsibility  as  the 
basic  agreement  between  the  supporting 
and  using  commands  for  management  and 
support  of  computer  resources.  Again, 
the  Initial  publication  of  the  weapon 
system  CRISP  will  probably  not  give 
specific  coverage  to  ATE. 


The  CRISP  is  written  as  a  part  of  and  In 
parallel  with  the  PMP.  It  is  prepared  by 
a  CRWG  consisting  of  representatives  of 
the  Implementing,  using  and  support 
commands.  The  composition  of  the  CRWG 
ensures  that  necessary  elements  of  the 
CRISP  are  Included  In  transfer  and  turn¬ 
over  agreements.  The  CRISP  is  a  living 
document  in  that  it  is  continuously 
updated  during  the  system  life  cycle. 
The  CRISP  and  its  updates  are  the 
responsibility  of  the  Program  Manager. 
The  weapon  system  CRISP  may  either  be 
updated  to  include  ATE  computer 
resources  after  the  ATE  system  is 
defined  or  a  CRISP  may  be  written  that 
is  devoted  to  ATE  computer  resources. 

During  the  formulation  of  the  initial 
weapon  system  CRISP  it  Is  doubtful  that 
ATE  engineering  would  be  an  active  parti¬ 
cipant.  As  the  system  develops  and  ATE 
is  identified  and  computer  resources  are 
allocated,  it  becomes  increasingly 
important  for  representation  on  the  CRWG 
since  the  purpose  is  to  provide  an  Inte¬ 
grated  support  plan  that  is  coordinated 
and  agreed  upon  by  all  active  partici¬ 
pants  in  the  acquisition  and  support 
phases. 

A  separately  prepared  CRISP  for  ATE  is 
the  preferred  approach.  It  should  be 
prepared  after  the  SERD  have  been  pub¬ 
lished.  The  separate  CRISP  provides  for 
a  singular  emphasis  on  the  management  of 
ATE  resources.  When  a  separate  CRISP  Is 
prepared,  it  is  mandatory  that  ATE  engi¬ 
neering  be  represented  on  the  CRWG  from 
its  Inception. 

In  either  case,  whether  a  separate  or 
confined  CRISP  Is  produced.  It  Is  some¬ 
times  difficult  to  get  the  sufficient 
support  from  all  affected  commands  for 
this  early  planning.  The  CRWG  chairman 
should  demand  the  appropriate  level  of 
support  from  each  command.  He  should  be 
supplied  with  experienced  personnel  that 
are  willing  and  able  to  spend  the  time 
and  effort  required  for  this  planning. 


Section  5.0  CONTRACT  DATA  REQUIREMENTS  LIST 


Selection  of  a  CDRL  is  of  vital  impor¬ 
tance  to  the  successful  management  of 
computer  program  development  activity. 
Computer  programs  are  a  significant  por¬ 
tion  of  TS  systems  and  ATE.  CDRL  selec¬ 
tion  differs  for  TS  and  ATE  computer  pro¬ 
grams  due  to  the  emphasis  in  the  acquisi¬ 
tion  process.  TS  systems  are  usually 
acquired  by  separate  contract.  ATE  is 
usually  acquired  as  part  of  a  weapon  sys¬ 
tem  contract  and  may  be  delayed  up  to 
several  years  before  serious  activity 
begins.  The  two  subjects  are  treated 
separately  in  this  section.  There  are 
some  document  commonalities.  They  will 
be  repeated  in  each  subsection  to  pro¬ 
vide  completeness  and  independence  for 
ATE  and  TS  discussion.  General  processes 
and  definitions  common  to  both  ATE  and 
TS  systems  are  described  in  the  first 
part  of  this  section. 

5.1  SELECTION  FACTORS 

Selection  of  a  satisfactory  CDRL  is  of 
vital  importance  for  life  cycle  con¬ 
siderations  of  ATE  and  TS  system  com¬ 
puter  programs.  Proper  documentation  is 
not  only  important  during  the  develop¬ 
ment  phase  but  also  in  the  operation  and 
support  phase  for  these  computer  pro¬ 
grams.  Contractor  documentation  provides 
the  link  between  the  development  contrac¬ 
tor,  the  implementing  command,  the  using 
command  and  the  support  command. 

Often,  lip  service  is  given  to  the 
recognition  of  the  importance  of  com¬ 
puter  program  documentation;  but  little 
time  is  actually  devoted  to  the  analysis 
and  selection  of  a  CDRL.  For  example,  it 
is  easy  to  say  that  documentation  is 
important,  then  scan  the  authorized  data 
list  for  DIDs  and  select  all  Items  that 
appear  applicable.  The  other  extreme  is 
to  be  so  cost  conscious  that  Important 
documentation  is  omitted  to  the  detri¬ 
ment  of  some  participants.  Too  little 
documentation  may  actually  increase  the 
total  life  cycle  costs  by  producing  an 
unreliable  computer  program  product  with 


many  errors  yet  undetected,  and  by 
causing  the  using  and  support  command  to 
either  purchase  the  documentation  at  a 
later  date  or  attempt  to  produce  the 
needed  data  themselves.  Providing  nee<‘<*d 
documentation  is  cost  effective.  Produc¬ 
ing  documentation  that  is  not  needed  is 
wasteful.  The  selection  of  a  CDRL  for 
ATE  and  TS  is  usually  made  by  the  imple¬ 
menting  command  for  the  using  and  sup¬ 
porting  commands.  Since  documentation  is 
expensive,  only  those  documents  that  are 
necessary  should  be  chosen.  Each  docu¬ 
ment  should  have  a  peculiar  purpose. 
When  selecting  a  CDRL,  one  must  always 
keep  in  mind  the  needs  for  documentation 
and  choose  those  documents  that  satisfy 
the  needs.  Computer  program  documenta¬ 
tion  should  provide  for  the  following: 

a.  Development  planning, 

b.  Identification  of  the  programs  to 
be  developed, 

c.  Identification  of  the  product  to 
be  delivered, 

d.  Testing, 

e.  Configuration  management, 

f.  Instructions  to  the  programmer  for 
use  of  the  computer  and  the  languages 
used,  and 

g.  User  and  maintenance  information. 

The  CDRL  selection  process  begins  with  a 
data  call  by  the  project  office  for  all 
affected  agencies.  Appropriate  DIDs  are 
chosen  from  the  DOD  5000. 19-L  document 
to  match  the  requirements  listed  above. 
Care  must  be  taken  that  specific  require¬ 
ments  for  each  ATE  or  TS  application  are 
considered.  The  list  of  DIDs  Is  exam¬ 
ined.  Each  DID  should  be  subjected  to 
the  following  questions: 

a.  Why  is  this  data  item  needed? 

b.  Who  will  use  it? 

c.  Are  any  data  items  on  the  list 
redundant? 

d.  When  will  they  be  required? 
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5.1.1  Identify  the  Need 

The  need  for  each  data  item  should  be 
identified.  Does  the  data  item  satisfy 
one  or  more  of  the  needs  listed  in  para¬ 
graph  3.1?  If  it  does  not,  then  it 
should  be  eliminated  unless  there  are 
some  other  compelling  reasons  for  its 
use. 

5.1.2  Identify  the  Users 

Identification  of  the  users  is  really  an 
extension  of  identifying  the  need.  The 
users  may  represent  more  than  one  com¬ 
mand.  Since  the  CDRL  is  usually  chosen 
by  the  implementing  command  for  other 
commands,  it  is  necessary  for  them  to  be 
cognizant  of  the  documentation  needed  by 
using  and  supporting  commands.  The  data 
items  must  be  examined  to  ensure  that 
the  users  needs  are  satisfied  by  the 
data  items.  If  not,  other  data  items 
should  be  considered  or  the  data  items 
should  be  tailored  to  satisfy  the  users 
need. 

5.1.3  Redundant  Data  Items 

The  DIDs  oh  the  proposed  list  should  be 
examined  to  determine  if  there  are  redun¬ 
dant  data  items.  Redundant  items  are  com¬ 
pared  to  determine  which  ones  best 
satisfy  the  document  user's  needs.  Redun¬ 
dancies  should  be  eliminated.  If  no  one 
DID  satisfies  all  needs,  consideration 
should  be  given  to  tailoring  the  DID  or 
to  writing  a  unique  DID  for  the  project. 
A  DID  can  be  tailored  to  combine 
material  from  several  DIDs  to  achieve  a 
desired  result. 

5.1.4  Identify  the  Time  Needed 

Documentation  is  of  little  use  if  it  is 
not  available  when  needed.  Normally, 
documentation  is  prepared  at  specific 
milestones,  meeting  specific  needs  and 
providing  a  record  of  the  software  devel¬ 
opment  status  at  that  time.  These  times 
should  be  identified,  and  analyzed  to 
assure  that  the  documentation  satisfies 


the  need  at  a  given  time.  These  need 
dates  can  then  be  included  in  the  CDRL. 

5.2  DATA  ITEM  DESCRIPTION 

The  DID  is  the  description  of  a  data 
item  required  to  be  furnished  by  a  con¬ 
tractor.  All  approved  DIDs  are  published 
in  DOD  5000. 19-L,  Acqui siting  Management 
Systems  and  Data  Requirements  Control 
List,  also  known  as  the  Authorized  Data 
List  (ADL).  The  DIDs  in  the  ADL  have 
been  authorized  for  procurement  under 
the  Contract  Data  Management  Program  as 
described  in  AFR  310-1,  Management  of 
Contractor  Data. 

Most  approved  DIDs  are  written  for 
general  application.  ATE  and  TS  system 
computer  programs  have  unique  require¬ 
ments  which  are  not  covered  by  the 
approved  DIDs.  Therefore,  tailoring  DIDs 
for  the  specific  application  is  neces¬ 
sary.  There  are  enough  differences  in 
ATE  and  TS  system  computer  program  docu¬ 
mentation  to  justify  the  generation  of 
unique  DIDs.  The  process  of  developing 
new  DIDs  is  time  consuming  and  will  not 
satisfy  current  needs  because  of  the 
amount  of  calendar  time  required  for 
authorization  (up  to  two  years),  but 
should  be  considered.  Tailoring  is  the 
method  that  can  be  used  for  each  appli¬ 
cation  and,  indeed,  should  be  encour¬ 
aged.  Even  if  unique  DIDs  are  written 
and  approved,  tailoring  is  still 
required  for  each  application,  though  to 
a  lesser  degree. 

Tailoring  is  ususally  accomplished  by 
using  a  DID  that  has  been  previously 
modified  for  a  similar  application  as  a 
baseline,  i.e.,  for  a  previous  ATE  or  TS 
acquisition.  After  the  unique  features 
of  the  new  applications  are  considered 
the  previously  modified  DID  can  be 
tailored  by  eliminating  superfluous 
requirements  and  adding  others  that  are 
missing.  The  resulting  DID  will  then  be 
tailored  to  satisfy  the  exact  require¬ 
ments  of  the  new  application. 
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General  tailoring  of  DIDs  has  been  done 
to  a  greater  degree  for  TS  computer  pro¬ 
grams  and  to  a  lesser  degree  for  ATE  com¬ 
puter  programs.  The  process  of  devel¬ 
oping  unique  DIDs  and  tailoring  standard 
DIDs  Is  described  In  AFR  310-1. 
Additional  Information  on  creating  new 
DIDs,  developing  unique  DIDs  and  modi¬ 
fying  DIDs  may  be  found  In  ASPR  16-828 
and  D0D-I-5010-12. 

When  a  unique  or  tailored  DID  Is  pre¬ 
pared,  It  should  be  included  with  the 
CDRL  package.  Standard  DIDs  need  only  to 
be  referenced  because  they  are  readily 
available  to  the  contractor.  The  modi¬ 
fied  DID  should  be  produced  on  the 
Authorized  Data  List,  DD  Form  1664  "Data 
Item  Description."  AFR  310-1  attachment 
2  specifies  the  instructions  for  pre¬ 
paring  the  form. 

5.3  DATA  CALL 

The  data  call  is  the  formal  procedure 
used  by  a  project  data  management 
officer  to  acquire  data  requirements 
from  participating  government  activi¬ 
ties.  ATE  and  TS  engineering  are  parti¬ 
cipants.  The  data  call  process  is 
illustrated  by  figure  5.3-1.  The  initial 
data  call  is  issued  following  project 
approval  at  DSARC1  when  work  on  a  SOW  is 
initiated.  Normally  this  is  a  weapon  sys¬ 
tem  SOW  for  ATE  and  the  TS  system  SOW 
for  TS  systems.  The  data  call  is  dis¬ 
tributed  to  the  participating  government 
organizations,  which  include  the  imple¬ 
menting,  using  and  support  commands. 
Many  times  the  implementing  command  acts 
for  the  using  and  support  commands  in 
this  matter. 

Each  participant  analyses  his  data 
requirements  in  accordance  with  the 
needs  described  in  paragraph  5.1  then 
selects  an  appropriate  DID  from  the 
authorized  data  list  (DOD  5000. 19-L). 
Often  times  a  DID  has  been  modified  for 
a  similar  application  and  can  be  chosen 
in  place  of  the  basic  DID.  The  selected 
DIDs  are  carefully  examined  to  determine 
their  applicability.  The  selected  DIDs 
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are  then  modified  as  necessary  to  satis¬ 
fy  the  exact  requirements  for  the  appli¬ 
cation.  Form  1664  "Data  Item  Descrip¬ 
tion"  is  completed  for  each  modified  DID 
as  described  in  Attachment  2  to  AFR 
310-1  "Management  of  Contractor  Data." 
The  suffix  /M  is  added  to  the  Data  Item 
Number  to  designate  its  modified  status. 
DID  forms  are  always  attached  to  the 
CDRL  when  the  DID  has  been  modified. 

When  all  data  items  have  been  selected 
and  DIDs  have  been  appropriately  modi¬ 
fied,  Form  1423  "Contract  Data  Require¬ 
ment  List"  is  completed.  Each  data  item 
is  identified  and  described  according  to 
instructions  contained  in  Attachment  3 
to  AFR  310-1.  In  general,  the  informa¬ 
tion  to  be  inserted  in  each  block  on 
Form  1423  is  adequately  described  in  the 
instructions.  However,  blocks  11,  12  and 
13  require  specific  dates.  Submittal 
dates  for  computer  program  documentation 
are  better  related  to  certain  milestones 
such  as  Preliminary  Design  Review  (PDR), 
Critical  Design  Review  (CDR),  Functional 
Configuration  Audit  (FCA),  Physical  Con¬ 
figuration  Audit  (PCA),  etc.  For  exam¬ 
ple,  a  preliminary  product  specification 
may  be  submitted  15  days  before  COR. 
Since  there  is  not  enough  space  in  these 
blocks,  block  16  should  be  used.  Care 
should  be  taken  that  the  correct  distri¬ 
bution  of  the  data  items  should  be  thor¬ 
oughly  researched  to  assure  those  need¬ 
ing  the  data  are  included  and  that  each 
organization  that  appears  on  the  list 
has  a  need  for  the  number  of  copies 
specified. 

The  completed  CDRL  is  then  reviewed  by  a 
data  requirement  review  board  for  all 
contracts  exceeding  one  million  dollars. 
The  project  data  management  offices  may 
approve  the  CDRL  for  lesser  contracts. 
The  review  board  may  consolidate  some 
items  and  eliminate  others.  Some  wording 
may  be  changed  in  the  review  process. 
Upon  approval  the  initial  CDRL  is 
established.  The  CDRL  review  process  is 
a  continuing  activity  and  is  repeated 
whenever  new  requirements  are  estab¬ 
lished.  It  is  also  vitally  important 
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Figure  5.3-1.  Data  Call  Process 


that  the  approved  CDRL  be  reviewed  by 
the  organizations  that  were  involved  in 
the  CDRL  generation  to  assure  that  their 
requirements  are  still  represented  in 
the  approval  list.  The  approved  CDRL  is 
then  included  with  the  completed  SOW  in 
the  RFP. 

TS  system  acquisitions  are  usually 
separate  contracts  and  the  data  call  is 
both  directly  applicable  and  current.  It 
is,  therefore,  relatively  easy  for  the 
TS  system  engineer  to  keep  abreast  of 
CDRL  activity.  On  the  other  hand,  the 
initial  ATE  CDRL  may  be  established 
years  before  ATE  is  seriously  considered 
and  it  is  part  of  a  much  larger  and  more 
expedient  procurement.  It  is  important 
that  ATE  documentation  considerations 
are  not  overlooked  even  at  the  initial 
data  call.  ATE  software  engineering 
should  respond  to  the  data  call  and  pro¬ 
vide  their  data  requirements.  Follow-up 
is  necessary  after  the  initial  CDRL  is 
published  to  assure  ATE  provisions  are 
included;  e.g. ,  during  the  consolidation 
of  similar  or  identical  DIDs,  words  may 
be  left  out  making  the  DID  inapplicable 
for  ATE  documentation.  Since  the  CDRL 
for  TS  systems  is  usually  produced  speci¬ 
fically  for  that  application  and  is  also 
more  timely  this  is  usually  not  a  prob¬ 
lem.  However,  nothing  should  be  taken 
for  granted  and  the  final  CDRL  must  be 
reviewed  before  the  RFP  is  released. 

5.4  DATA  ACCESSION  LIST 

Frequently  contractors  will  prepare  docu¬ 
mentation  for  their  own  use  and  in  their 
own  formats  that  may  be  useful  to  the 
Air  Force  but  is  not  included  in  the 
CDRL.  The  use  of  the  Data  Accession  List 
technique  requires  contractors  to  pro¬ 
vide  a  list  of  their  Internal  data  they 
are  generating  for  their  own  use  In  per¬ 
formance  of  the  contract.  The  Air  Force 
can  acquire  this  data  In  the  contrac¬ 
tor's  format,  however.  It  is  not  subject 
to  Air  Force  approval  or  a  delivery 
schedule.  These  data  can  thus  be 
acquired  at  little  or  no  additional 


cost.  The  DID  describing  the  data  Acces¬ 
sion  List  is  DI-A-3027/M-128. 

The  use  of  this  technique  Is  not 
intended  to  replace  the  careful  con¬ 
sideration  of  a  CDRL  and  placing  those 
requirements  on  a  contractor.  Reliance 
on  a  Contractor  Data  Accession  List  Is  a 
dangerous  practice  and  should  be 
avoided.  Data  requirements  should  arise 
from  the  eventual  users  of  the  docu¬ 
ments,  not  the  contractor.  Contractor 
documents  may  be  outstanding  on  ocasslon 
but  they  have  no  contractual  authority 
as  to  content,  quality,  or  the  time  they 
are  to  be  delivered.  The  Data  Accession 
List  may  not  even  contain  all  the  docu¬ 
ments  the  contractor  has  prepared  with 
the  Air  Force  having  no  way  of  knowning 
what  documents  are  readily  available.  It 
should  be  used  only  as  supplementary 
data  that  will  be  useful  to  the  Air 
Force.  When  contractor  data  are  found  to 
be  useful,  their  inclusion  should  be  con¬ 
sidered  for  inclusion  in  future  computer 
program  acquisitions. 

5.5  DESCRIPTION  OF  KEY  DOCUMENTS 

Identification  of  the  key  contractor- 
prepared  documents  for  the  acquisition 
of  TS  computer  programs  is  based  on  DODD 
5000.29,  Management  of  Computer 
Resources  for  Major  Defense  Systems, 
which  states  "Defense  system  computer 
resources,  including  both  computer  hard¬ 
ware  and  computer  software  will  be  speci¬ 
fied  and  treated  as  configuration 
items."  Satisfying  this  requirement 
implies  satisfaction  of  the  needs  for 
documentation  specified  in  paragraph 
3..1.  This  section  describes  the  key  docu¬ 
ments  required  for  the  acquisition  of 
ATE  and  TS  software.  The  role  each  of 
these  documents  plays  is  described  in 
its  relation  to  the  system  acquisition 
and  the  computer  program  acquisition.  In 
essence,  the  following  paragraphs  are  a 
recommendation  for  a  CDRL  for  TS  and  ATE 
computer  programs.  It  is  based  on  cur¬ 
rent  Air  Force  practices  and  industry 
experience.  Since  there  is  a  consider- 


able  difference  between  the  acquisition 
processes  for  ATE  and  TS  software,  the 
discussion  will  be  separated  Into  two 
separate  paragraphs.  Paragraph  5.5.1 
will  cover  contractor-prepared  documents 
for  TS  computer  programs  and  Paragraph 
5.5.2  will  cover  ATE  computer  programs. 

5.5.1  TS  Documentation 

The  following  subsections  provide 
descriptions  of  key  documents  for  TS  sys¬ 
tem  computer  programs,  a  summary  of  the 
documentation  required  and  a  checklist, 
(Table  5.5-1),  for  selecting  the  CDRL 
for  TS  computer  programs  Is  provided. 

TS  systems  are  often  acqul red  as  a 
single  Cl  and  specified  accordingly.  The 
TS  system  specification  Is  prepared  by 
the  Air  Force  as  described  in  paragraph 
4. 1.5. 3.  It  includes  both  hardware  and 
computer  program  requirements.  If  the 
system  is  composed  of  multiple  CIs, 
separate  specifications  will  be  prepared 
by  the  Air  Force.  These  specifications, 
prepared  by  Air  Force  TS  engineering  per¬ 
sonnel,  will  be  the  required  development 
specifications.  They  will  be  augmented 
by  the  contractor  technical  proposal. 

The  TS  Computer  Program  Development 
Specification  consists  of  the  TS  System 
Specification  and  the  contractor  pro¬ 
posal  and  Is  not  specifically  identified 
In  the  CDRL.  All  other  documentation  pur¬ 
poses  are  satisfied  through  contractor 
prepared  documents.  Some  of  the  docu¬ 
ments  supporting  the  computer  programs 
are  written  at  the  system  level  and  some 
are  written  specifically  for  computer 
programs.  The  following  key  documents, 
prepared  by  the  contractor,  satisfy  the 
documentation  needs  specified  In  para¬ 
graph  3.1.  The  list  Is  based  on  current 
acquisition  practices  an  contractor 
experience. 

Computer  Program  Unique  Documents 
Computer  Program  Development  Plan 
Interface  Design  Description 
Computer  Program  Product  Specifica¬ 
tion 


Training  Equipment  Computer  Program 
Documentation 

Version  Description  Document 
TS  System  Documentation 

Contractor  Technical  Proposal 
Test  Plans  and  Procedures 
Test  Reports 
Configuration  Index 
Change  Status  Report 
Engineering  Change  Proposal 
Specification  Change  Notice 
Data  Accession  List 

If  computer  programs  are  broken  out  as 
CPCIs,  separate  test  plans  and  test  pro¬ 
cedures  will  be  prepared  for  each  CPCI. 
The  remainder  of  the  system  documents 
will  still  be  addressed  at  the  system 
level. 

5. 5. 1.1  Contractor  Technical  Proposal. 
The  contractor  proposal  Is  the  bidders 
response  to  the  RFP.  It  contains  the 
technical  data  proposed  by  the  bidders 
to  develop,  build  and  deliver  TS  sys¬ 
tems.  Included  in  the  proposal  package 
are  a  CPDP  for  the  software  part  of  the 
TS  system,  and  a  CMP  for  the  entire  TS 
system. 

The  technical  proposal  provides  addi¬ 
tional  contractor-prepared  material  that 
is  not  included  In  the  system  specifica¬ 
tions.  This  material  Is  Intended  to 
refine  the  requirements  to  demonstrate 
the  contractor's  understanding  of  the  TS 
system  and  to  give  him  a  competitive 
edge.  Therefore,  It  fills  out  the 
requirements  as  the  contractor  under¬ 
stands  them.  For  this  reason  the  techni¬ 
cal  proposal  will  usually  become  a  con¬ 
tract  document,  albeit,  the  lowest  level 
of  contract  requirements;  l.e. ,  all 
other  requirements  take  precedence  If 
there  are  conflicts. 

5. 5. 1.2  Computer  Program  Development 
Plan  (UDI-S-3911/ASD).  The  CPDP  Is  one 
of  the  major  computer  resource  planning 
documents.  AFR  800-14,  Volume  II, 
Acqulsltlng  and  Support  Procedures  for 
Computer  Resources  In  Systems,  requires 
a  CPDP  for  the  acquisition  of  computer 
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Table  5.5-1.  Trainer  Simulator  CDRL  Checklist  (Sheet  1  of  3! 


Does  the  CDRL  specify  the  following  documents  or  their  equivalent? 

(NO  DID)  Interface  Design  Description 

DI-E-3120A/M1  Computer  Program  Product  Specification 
DI-H-3277/M3  Training  Equipment  Computer  Program  Documentation 
DI-E-3121  Version  Description  Document 

DI-T-3703  Category  I  Test  Plans/Procedures  (Computer  Program) 

DI-T-3717  Category  I  Test  Report  (Computer  Program) 

DI-E-3108  Configuration  Management  Plan 

DI-E-3122  Configuration  Index 

DI-E-3123  Change  Status  List 

DI-E-3134  Specification  Change  Notice 

DI-A-0327  Data  Accession  List/Internal  Data 


2.  Have  DID's  been  tailored  to  satisfy  all  requirements  for  the  specific 
application? 

3.  Has  each  DID  been  examined  to  ensure  it  satisfies  the  requirements  of  the 
specific  application? 

4.  Have  appropriate  DID's  been  modified  for  use  of  top  down  structured 
programming  techniques  Including  use  of  program  design  language? 

5.  Have  all  document  users  been  identified?  Have  they  been  consulted? 

6.  Has  time  been  established  for  review  and  delivery  for  each  document?  Are 
schedules  related  to  specific  milestone  events  such  as  PDR,  CDR,  etc. 

7.  Do  the  DID's  for  the  following  documents  contain  provision  for  key  items 
as  shown  below? 


Interface  Design  Description 


Interfaces  specified  separately  or  adequately  described  In  the 
product  specification  DID 

External  Interface  descriptions  Including 

Data  formats 
Frequency 

Methodology  for  passing  and  receiving  data 


Tabh  5.5-1.  Train*  Simulator  CD f II  ChacUkt  (Shaat  2  of  3) 


c.  Internal  interface  descriptions  including 

Data  base  structure 
Methodology  for  information  transfer 
Data  types 
Data  files 
Size 

Set/used  information 
Product^pecification 

a.  Complete  description  of  computer  program  including 

Descriptive  narrative 
Logic  flows 
Program  listings 

b.  Mathematical  model  description 

c.  Computer  timing  and  sizing  estimate 

d.  Top  down  structured  programming  techniques 

e.  Program  design  language 

Test  Equipment  Computer  Program  Documentation 

a.  Program  design  conventions  and  philosophy 

b.  Operating  instructions 

Initiating  operation 
Maintaining  operation 
Restart 

c.  System  generation  procedures 

d.  Programming  manuals  for  each  language/computer  combination 

e.  Programmer  note  book 

f.  Top  down  structured  programming  techniques 

g.  Program  design  language  philosophy 


Table  5.5-1.  Trainer  Simulator  CDRL  Checklist  (Sheet  3  of  3) 
Test  Pl*an/ Procedure 

h.  Test  plans  for  all  levels  of  computer  program  testing 

i.  Delivery  of  "as-run"  test  procedures 


j.  Top  down  integration  techniques 


programs.  It  Is  normally  prepared  by  a 
contractor  for  the  developing  agency  as 
part  of  the  proposal  package. 

The  CPDP  applies  to  all  phases  of  the 
software  development  cycle;  It  Is  of 
particular  Importance  to  the  analysis, 
design,  coding  and  checkout,  and  test 
and  Integration  phases.  It  defines  the 
contractor's  overall  plan  for  developing 
computer  programs  and  necessary  support¬ 
ing  resources.  The  plan  includes  identi¬ 
fication  of  the  computer  program  pro¬ 
ducts  to  be  delivered,  the  development 
schedule  and  related  documentation.  It 
includes  a  description  of  the  devel¬ 
opment  organization;  responsibilities 
for  design,  implementation,  testing  and 
Integration;  hardware  and  facilities 
required;  and  procedures  for  managing 
and  controlling  all  aspects  of  devel¬ 
opment.  The  CPDP  should  be  used  by  the 
contractor  to  describe  his  procedures 
for  controlling  design  changes  prior  to 
the  establishment  of  configuration 
management  baselines.  It  should  address 
the  reporting  and  management  of 
discrepancies  discovered  in  testing, 
responsibilities  for  failure  analysis 
and  correction,  retesting  and  the  con¬ 
trol  of  both  source  and  object  code.  In 
addition,  the  CPDP  should  describe  the 
contractor's  approach  to  performance 
estimation  and  refinement  of  the  esti¬ 
mates  in  terms  of  responsibilities, 
resource  allocation  and  relationships  to 
the  development  schedule. 

Since  the  CPDP  Is  prepared  as  a  part  of 
the  contractor  proposal,  it  provides  an 
additional  factor  In  the  proposal  eval¬ 
uation  process.  It  Is  also  a  common  prac¬ 
tice  to  place  the  CPDP  "on-contract" 
thus  the  contractor  is  obliged  to 
observe  the  procedures,  controls  and 
methods  defined  in  It.  There  are  some 
drawbacks  to  the  practice  such  as  being 
contractually  committed  to  a  given 
organizational  structure  or  to  schedules 
that  may  prove  unrealistic.  These  draw¬ 
backs  not  only  affect  the  contractor  but 
the  Air  Force  as  well,  having  to  nego¬ 
tiate  new  organizational,  structures, 
schedules,  etc. 


Since  the  CPDP  Is  prepared  as  a  part  of 
the  contractor  proposal.  It  Is  not 
Included  In  the  CDRl.  However,  it  Is 
Included  here  as  a  contractor-prepared 
document  that  Is  required  for  TS  com¬ 
puter  program  development.  The  CPDP  is 
called  out  In  the  SOW  and  should  be 
explained  in  detail  In  the  IFPP.  There 
are  several  different  DIDs  describing 
CPDPs.  Four  of  these  are  identified 
below: 

a.  Dl-A-5239 

b.  DI-S-3591  A/M 

c.  UDI-S-3911/ASD 

d.  UDI-E-695/ESD 

These  DIDs  are  similar,  but  all  differ 
somewhat  from  the  content  specified  in 
AFR  800-14  Volume  II.  Each  CPDP  is 
designed  for  a  given  application  and  the 
DID  should  be  specifically  tailored  to 
that  application.  If  the  CPDP  is  to  be 
placed  on-contract,  great  care  must  be 
taken  to  tailor  the  DID  in  such  a  way 
that  it  satisfies  the  objective  of 
committing  the  contractor  to  a  given 
development  process,  but  does  not  con¬ 
tain  unnecessary  constraints.  The  CPDP 
must  be  able  to  accomodate  changes  in 
requirements  during  the  development 
period.  Thus,  the  contractor  should  be 
directed  (in  the  SOW)  to  update  the  CPDP 
at  specified,  appropriate  intervals  such 
as  PDRs  and  CDRs. 

5. 5. 1.3  Interface  Design  Description 
(No  Applicable  DID).  Interface  design  is 
of  more  than  passing  interest  in  com¬ 
puter  program  development.  In  a  project 
with  more  than  one  or  two  programmers 
involved,  exact  interface  descriptions 
between  programs  and  between  program  com¬ 
ponents  become  of  prime  importance  in 
communications  between  programmers. 
Interface  design  is  a  computer  program 
system  engineering  task.  It  is  the  frame¬ 
work  within  which  the  various  programs 
and  components  exchange  Information. 
Data  base  design  Is  Integral  to  the 
Interface  definitions. 

Interfaces  can  be  considered  at  two 
levels:  (1)  Interface  between  a  computer 


program  and  external  devices  through  I/O 
channels  and  (2)  Interfaces  between  com¬ 
puter  programs  and  between  computer  pro¬ 
gram  components.  Both  are  of  vital 
Importance.  Computer  program  Internal 
Interfaces  affect  only  the  computer  pro¬ 
gram  designers;  external  Interfaces 
affect  other  design  organizations. 

Experience  has  shown  that  Interface 
definitions  tend  to  be  overlooked  when 
they  are  not  emphasized  In  the  develop¬ 
ment  process.  This  lack  of  definition  of 
the  Interfaces  leads  to  confusion  among 
programmers  and  between  programmers  and 
other  system  designers.  When  given 
proper  emphasis  by  preparing  separate 
Interface  documents,  a  better  definition 
has  resulted.  It  has  been  easier  to 
review,  and  interface  information  has 
been  easier  to  locate  and  use  for 
troubleshooting  and  maintenance.  A  TS 
system  should  combine  these  into  a 
single  Interface  document  for  both 
Internal  and  external  Interfaces. 

The  interface  design  description  docu¬ 
ment  Includes  detailed  descriptions  of 
all  external  Interfaces.  The  exact  for¬ 
mat,  frequency  and  methodology  for 
passing  and  receiving  data  are  included. 
Internal  Interface  descriptions  Include 
the  data  base  structure,  methodology  for 
passing  and  receiving  data  and  detailed 
descriptions  of  files  and  of  the  indivi¬ 
dual  data  elements.  Included  are  the 
data  type;  e.g. ,  arrays,  items,  files, 
etc.;  characteristics;  e.g.,  floating 
point,  integer.  Binary  Coded  Decimal 
(BCD),  binary,  etc.;  identifier;  size, 
e.g.,  number  of  bits,  bytes,  words; 
identifications  of  programs  that  set  or 
use  the  data  Items;  and  a  description  of 
the  Item.  The  organization  of  data  items 
Into  files  or  other  d^ta  structures  Is 
also  shown  with  the  names,  dimensions 
and  other  distinguishing  characteristics 
of  data  base  flies.  In  some  cases, 
depending  on  the  support  software  avail¬ 
able,  much  of  the  Information  required 
for  these  data  base  files  can  be  pro¬ 
duced  automatically. 


In  current  TS  system  acquisitions,  these 
Interface  data  are  Included  as  part  of 
the  Computer  Program  Product  Specifica¬ 
tion.  There  are  no  DIDs  addressing  com¬ 
puter  programs  Interface  design  descrip¬ 
tions.  Therefore,  a  unique  DID  would 
have  to  be  generated  and  submitted  to 
the  Command  Data  Mangement  Office  for 
approval  by  the  Command  Contractor  Data 
Management  Review  Board.  It  must  be 
approved  before  It  can  be  placed  on  con¬ 
tract.  In  the  meantime,  the  correspond¬ 
ing  sections  In  the  Computer  Program  Pro¬ 
duct  Specification  can  be  strengthed  by 
tailoring  the  existing  DID  or  by  locat¬ 
ing  a  DID  that  primarily  addresses  Inter¬ 
face  definition  and  tailor  It  for  TS  com¬ 
puter  program  interfaces. 

The  Interface  Design  Description  Docu¬ 
ment  is  prepared  by  the  contractor  in 
parallel  with  the  Computer  Program  Pro¬ 
duct  Specification.  A  preliminary  draft 
Is  prepared  at  the  TS  computer  program 
PDR  and  a  complete  draft  at  the  TS  com¬ 
puter  program  CDR.  It  Is  delivered  to 
the  Air  Force  at  the  PCA. 

5. 5. 1.4  Computer  Program  Product  Speci¬ 
fication  (DI-E-3120A/M1).  The  Computer 
Program  Product  Specification  estab¬ 
lishes  the  detailed  technical  descrip¬ 
tion  of  the  TS  computer  programs  to  be 
delivered  under  terms  of  the  contract. 
It  Includes  a  complete  description  of 
the  TS  computer  programs  Including  pro¬ 
gram  logic  flows  and  program  listings 
supported  by  appropriate  narrative.  A 
preliminary  draft  Is  prepared  by  the 
contractor  for  the  PDR,  a  complete  draft 
is  prepared  for  the  CDR  and  It  Is 
delivered  at  the  PCA.  When  approved  It 
establishes  the  configuration  management 
product  baseline.  Changes  from  this  time 
on  require  Air  Force  approval.  Prior  to 
delivery,  the  product  specification  pro¬ 
vides  a  means  for  baseline  control, 
Internal  to  the  contractor's  organiza¬ 
tion,  for  design  reviews  and  for  Informa¬ 
tion  exchange  among  programmers. 

The  computer  program  product  speclfica- 
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tlon  DID  for  TS  systems  Is 
DI-E-3120A/M1.  This  Is  a  version  of  the 
basic  DID  that  has  been  tailored  for  TS 
computer  programs.  In  essence.  It  con¬ 
sists  of  the  first  two  sections  of 
DI-H-3277  which  are  the  mathematical 
model  documentation  and  the  computer  pro¬ 
grams  system  description.  Interface 
design  descriptions  are  addressed  but  as 
discussed  in  paragraph  5. 5. 1.3,  should 
be  either  expanded  or  removed  and 
included  in  a  separate  document. 

For  most  computer  programs  the  product 
specification  is  a  large  document.  It 
should,  therefore,  be  divided  into  a 
number  of  volumes  to  suit  the  specific 
application.  Separate  volumes  can  be  pre¬ 
pared  for  the  overall  software  system 
level  design,  for  major  TS  functions  and 
for  the  interface  definitions.  The  sys¬ 
tem  level  column  is  prepared  as  a  par¬ 
tial  product  specification  draft  to  sup¬ 
port  the  PDR  and  would  become  a  contrac¬ 
tor  baseline  from  that  time  on.  The 
major  functions  and  interface  volumes 
are  prepared  in  draft  form  for  the  CDR. 

The  product  specification  must  be  kept 
current  if  it  is  to  be  of  use  to  either 
the  contractor  or  the  developing  agency. 
The  product  specification  draft  provided 
at  CDR  should  represent  a  complete 
design.  Computer  code  is  then  generated 
in  accordance  with  the  design.  Changes 
are  inevitable  and  are  expected.  The 
contractor  should  describe  his  approach 
to  handling  changes  and  keeping  the 
product  specification  draft  current  in 
the  CPDP.  Detailed  logic  flows  and  com¬ 
puter  timing  and  sizing  estimates  in 
particular  remain  fluid  and  require 
effort  to  keep  current.  After  delivery 
to  the  Air  Force  it  is  even  more  diffi¬ 
cult  to  maintain  currency  due  to  the 
length  of  time  to  approve  changes 
through  change  board  action. 

There  are  three  parts  of  a  product  speci¬ 
fication  for  which  a  ready  correlation 
should  exist:  the  mathematical  model, 
the  logic  flows  and  the  program  list¬ 
ings.  Care  should  be  taken  to  assure 


that  these  three  parts  can  be  easily 
correlated  by  clearly  annotating  these 
parts.  Program  listings  represent  the 
actual  executable  computer  program  code, 
the  logic  flows  represent  the  design  and 
the  mathematical  models  represent  the 
theory  behind  the  design.  The  use  of 
top-down  structured  programming  techni¬ 
ques  may  include  the  use  of  a  program 
design  language  (PDL).  The  logic  flows 
may  be  expressed  in  the  PDL  and  are  used 
in  place  of  the  traditional  flow  charts. 
The  product  specification  DID  should  be 
tailored  to  provide  for  this  possi¬ 
bility. 

5. 5. 1.5  Training  Equipment  Computer 
Program  Documentation  (DI-H-3277/M3). 
Training  Equipment  Computer  Program 
documentation  is  a  composite  of  three 
documents:  (1)  a  user's  guide,  (2)  a 
programmer's  notebook,  and  (3)  a  com¬ 
puter  programming  manual.  Training 
Equipment  Computer  Program  Documentation 
is  prepared  by  the  contractor  and 
delivered  to  the  Air  Force  at  the  PCA. 
The  controlling  DID  is  DI-H-3277/M3. 
This  is  a  tailored  version  of  the  basic 
DID.  It  is  to  be  used  in  conjunction 
with  the  product  specification  described 
in  paragraph  5. 5. 1.4.  The  first  two 
sections  of  the  basic  DID,  Mathematical 
Model  Documentation  and  Computer  Program 
systems  Description,  are  removed  from 
this  document  and  included  in  the  pro¬ 
duct  specification  and  a  section.  Com¬ 
puter  Vendor  Programming  Manuals,  is 
added.  The  purpose  of  training  equipment 
documentation  is  to  augment  the  product 
specification  and  to  provide  all  techni¬ 
cal  data  required  for  TS  computer  pro¬ 
gram  maintenance  and  operation.  The  docu¬ 
mentation  may  be  included  in  a  single 
volume  or  may  be  divided  into  three 
volumes  depending  on  the  size  and 
expected  use  of  the  section.  The  three 
sections  are  described  in  the  following 
paragraphs. 

a.  Computer  Program  Users  Guide  -  The 
users  guide  section  provides  a  descrip- 
tion  of  how  the  computer  programs  were 
designed.  It  describes  the  general 


approach,  methods  employed  and  standards 
used  to  produce  the  product  specifica¬ 
tion  and  the  computer  program.  In  short, 
it  covers  most  of  the  areas  needed  for 
computer  program  maintenance.  However, 
user  information  regarding  usage  instruc¬ 
tions  (how  to  use  each  specific  func¬ 
tion),  computer  operating  instructions 
and  system  generation  instructions  as 
described  in  DI-M-3410  are  not  included 
in  DI-H-3277/M.  The  DID  should  be  modi¬ 
fied  to  include  these  items.  The  first 
two  items  are  necessary  for:  initiating 
the  computer  program  operation,  maintain¬ 
ing  its  operation,  and  terminating  and 
restarting  program  operation.  System 
generation  procedures  are  necessary  for 
program  maintenance  when  changes  have 
been  introduced  and  a  new  computer  pro¬ 
gram  system  must  be  generated.  These 
procedures  also  provide  a  means  for 
quality  assurance  inspection  of  the  sys¬ 
tem  generation  process.  If  top-down 
structured  program  (TDSP)  techniques  are 
used,  the  TDSP  philosophy  should  be  pro¬ 
vided.  If  a  PDL  is  used  in  place  of  the 
traditional  flow  charts,  the  PDL 
philosophy  should  be  included  to  enable 
program  maintenance  personnel  to 
understand  the  specific  techniques  used. 

b.  Computer  Programmer  Notebook.  This 
section  is  an  informal  collection  of  the 
programmer  notes  to  explain  nonstandard 
approaches  to  certain  design  or  coding 
problems.  It  also  provides  a  record  of 
the  problems  encountered  during  coding 
and  debug.  This  section  can  be  of  signi 
ficant  value  for  program  maintenance  as 
it  explains  some  of  the  apparent  altera¬ 
tions  in  coding  techniques. 

c.  Computer  Vendor  Proqramni ng 
Manuals.  The  purpose  of  this  section  is 
to  provide  programming  instructions  for 
a  specific  computer  and  a  specific  pro¬ 
gramming  language.  A  section  is  required 
for  each  language  and/or  computer  used. 
It  is  approximately  the  equivalent  of 
DI-M-3411  with  the  exception  that  It 
does  not  have  provisions  for  a  situation 
In  which  only  limited  or  no  manuals  are 
available  from  the  computer  vendor.  The 


DID  should  be  modified  to  Include  this 
provision.  Availability  of  computer 
program  documentation  should  be  a  part 
of  the  computer  vendor  selection  pro¬ 
cess.  If  the  required  manauls  are  not 
available,  information  is  still  required 
and  should  be  provided  by  the  contractor 
in  accordance  with  the  aforementioned 
DID.  Consideration  should  be  given  to 
replacing  Training  Equipment  Computer 
Program  Documentation  by  a  Users  Guide 
DI-M-3410  and  a  Computer  Programming 
Manual  DI-M-3411.  These  DIDs  already 
contain  the  missing  parts  as  described 
above.  These  DIDs  should  be  tailored  to 
include  the  desirable  parts  of 
DI-H-3277/M3.  This  would  make  the  CDRL 
more  like  other  computer  program  acquisi¬ 
tions  and  therefore,  simplify  a  transi¬ 
tion  from  other  software  acquisitions  to 
TS  computer  programs. 

5. 5. 1.6  Version  Description  Document 
(DI-E-3121).  The  purpose  of  the  VDD  is 
to  identify  the  exact  configuration  of 
TS  computer  programs  and  the  interim 
changes  thereto.  It  is  used  to  identify 
each  version,  and  accordingly,  accom¬ 
panies  each  version  of  the  TS  computer 
program  and  each  release  of  an  interim 
change.  The  VDD  is  prepared  by  the  TS 
contractor  in  accordance  with  DI-E-3121. 
The  DID  essentially  refers  to 
MIL-STD-483  for  contents  description. 
The  VDD  is  delivered  to  the  Air  Force  at 
PCA  and  is  an  exact  description  of  the 
TS  software  configuration  delivered  at 
that  time. 

Many  different  versions  of  computer  pro¬ 
grams  are  generated  and  used  during  the 
installation  and  operational  phases. 
Several  of  these  versions  may  in  use 
simultaneously.  Configuration  management 
of  the  evolution  of  computer  programs 
can  be  a  monumental  problem  if  there  is 
no  means  for  Identifying  the  separate 
versions.  Basically,  a  new  version  is 
created  whenever  any  change  is  made.  The 
VDD  provides  the  same  function  as  a  top 
level  hardware  drawing  In  that  it  identi¬ 
fies  the  exact  version  of  each  component 
that  makes  up  the  computer  program.  The 
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exact  design  and  code  changes  to  a  speci¬ 
fic  version  of  a  program  component  are 
described  In  the  product  specification 
complete  with  logic  flow  and  program 
listings;  however,  a  composite  program 
configuration  consisting  of  a  specific 
version  of  program  components  Is  very 
difficult  If  not  impossible  to  Identify 
the  exact  configuration  all  the  software 
tools  required  to  generate  the  deliver¬ 
able  computer  programs.  This  includes 
the  version  Identification  of  compilers, 
assemblers,  link  editors  and  all  other 
software  tools  used  in  the  process. 

5. 5. 1.7  Test  Plans  and  Procedures 
(DI-T-3703).  Validation  of  TS  computer 
programs  is  usually  performed  as  part  of 
the  TS  system  validation.  Therefore, 
separate  test  plans  and  procedures  are 
not  prepared  for  computer  program  vali¬ 
dation  unless  computer  programs  are 
developed  as  CPCIs.  Test  plans  and  pro¬ 
cedures  are  prepared  by  the  TS  system 
contractor  as  specified  by  DI-T-3703. 

Test  plans  are  prepared  in  accordance 
with  Section  4  of  the  TS  system  speci¬ 
fication.  The  initial  draft  of  the  plan 
should  be  available  for  review  at  the 
PDR,  the  final  version  at  CDR.  Test 
plans  should  show  the  entire  test  plan 
leading  to  acceptance  of  the  TS  system. 
This  includes  programmer  level  tests, 
computer  program  integration  tests, 
tests  of  the  computer  programs  indepen¬ 
dent  from  the  TS  software  and  TS  system 
level  tests. 

Test  procedures  are  prepared  for  each 
individual  qualification  test  and 
specify  the  test  process  including 
identification  of  the  equipment  con¬ 
figuration,  equipment  set  up,  computer 
program  configurations,  step-by-step 
procedures  and  expected  results.  The 
contractor  may  provide  internal  test 
procedures  for  lower  level  tests  leading 
to  qualification.  These  procedures  are 
not  required  to  be  delivered  by  con¬ 
tract,  but  may  be  obtained  using  the 
data  accession  list  techniques  (para¬ 
graph  5.4). 


Test  plans  may  be  prepared  as  a  separate 
volume  since  they  are  prepared  in 
advance  of  test  procedures.  In  fact, 
each  separate  test  procedure  could  be 
prepared  as  separate  volumes  depending 
on  the  size  of  the  tests.  Test  proce¬ 
dures  for  formal  qualification  and 
acceptance  tests  must  be  approved  by  the 
Air  Force  prior  to  testing.  Changes  made 
during  a  test  must  be  recorded  in  the 
test  procedure  that  is  delivered  at  FCA. 
This  will  describe  the  test  procedure 
actually  performed. 

5. 5. 1.8  Test  Reports  (DI-T-3717).  Test 
reports  are  required  for  describing  the 
results  obtained  from  the  qualification 
test  procedures  described  in  paragraph 
5. 5. 1.7.  They  are  prepared  by  the  con¬ 
tractor  in  accordance  with  DI-T-3717. 
They  should  be  delivered  within  a  speci¬ 
fied  time  period  following  the  test  com¬ 
pletion.  The  delivery  should  be  speci¬ 
fied  in  block  12  of  the  CDRL  Form  1423. 
The  results  are  used  as  part  of  the  FCA 
process.  Delivery  of  lower  level  test 
reports  may  be  obtained  from  the  contrac¬ 
tor  using  the  data  accession  list  techni¬ 
que  (paragraph  5.4). 

5. 5. 1.9  Configuration  Management  Plan 
(DI-E-3108).  The  CMP  is  prepared  by  the 
contractor  to  describe  his  assignment  of 
organizational  responsibility  and  the 
procedures  used  in  the  accomplishment  of 
the  specific  configuration  management 
requirement  as  stated  in  the  SOW.  The 
CMP  is  prepared  as  part  of  the  contrac¬ 
tor  proposal  and  often  becomes  contrac¬ 
tual  after  contract  award  and  approval 
by  the  Air  Force.  Even  though  the  CMP  is 
prepared  for  the  TS  system,  it  should 
contain  provisions  for  configuration 
management  of  computer  programs.  Proce¬ 
dures  unique  to  computer  programs  should 
be  described  in  the  CPDP  and  referenced 
by  the  CMP.  The  CW>  is  prepared  in  accor¬ 
dance  with  DI-E-3108  and  MIL-STD-483. 

5.5.1.10  Configuration  Index 
(DI-E-3122).  The  configuration  index  for 
a  computer  program  is  part  of  the  inte¬ 
grated  approach  to  configuration  manage- 
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ment  In  accordance  with  Appendix  VIII  of 
MIL-STD-483,  Configuration  Management 
Practices  for  System,  Equipment,  Muni¬ 
tions  and  Computer  Programs.  It  provides 
a  record  of  the  current  status  of  speci¬ 
fications  and  selected  additional  docu¬ 
ments,  such  as:  test  plans,  procedures 
and  reports,  users  manuals  and  VDDs, 
which  depend  for  Item  content  on  the  TS 
system  configuration.  Document  status  Is 
summarized  by  dates  of  Issue,  document 
numbers  and  titles,  ECPs,  SCN,  and  revi¬ 
sion  Identifier  associated  with  each 
Issue.  Additionally,  It  contains  a  sum¬ 
mary  record  of  milestones  pertaining  to 
the  TS  development,  audit  and  qualifica¬ 
tion.  It  Is  prepared  by  the  contractor 
as  soon  as  the  documents  it  contains  are 
released. 

5.5.1.11  Change  Status  List 
(DI-E-3123).  The  Change  Status  List  is 
supplementary  to  the  configuration 
index.  It  details  the  status  of  all  pro¬ 
posed  changes  to  the  trainer  simulation 
system  for  which  the  contractor  is 
responsible  and  for  which  existing  docu¬ 
mentation  Is  listed  in  the  Configuration 
Index.  It  contains  a  list  of  each  succes¬ 
sive  ECP,  by  number,  prepared  against 
the  TS  system,  with  a  brief  indication 
of  the  status  of  the  ECP;  and  a  detached 
summary  of  the  status  information  for 
each  ECP  which  is  currently  active.  It 
is  prepared  by  the  contractor  in  accor¬ 
dance  with  DI-E-3123  and  MIL-STD-483.  It 
is  a  continuing  record  of  the  status  of 
proposed  and  approved  changes. 

5.5.1.12  Engineering  Change  Proposals 
(DI-E-3128).  The  ECP  is  the  vehicle  used 
to  prepare,  process  and  Incorporate 
Class  I  engineering  changes  to  the  appli¬ 
cable  configuration  management  baseline, 
i.e. ,  development  specification  or  pro¬ 
duct  specification.  It  is  usually  pre¬ 
pared  by  the  contractor  and  must  be 
approved  by  the  government.  It  is  pre¬ 
pared  when  a  change  Is  considered 
appropriate  and  In  accordance  with 
DI-E-3128. 


5.5.1.13  Specification  Change  Notice 
(DI-E-3134).  The  SCN  Identifies  a  pro¬ 
posed  change  to  a  contractually  appll- , 
cable  specification  and,  after  approval, 
provides  a  record  of  the  change  and  the 
associated  ECP.  The  SCN  Is  prepared  by 
the  TS  system  contractor  whenever  an 
engineering  change  is  proposed,  and  Is 
Included  as  part  of  the  ECP  package.  The 
SCN  Is  prepared  In  accordance  with 
DI-E-3134  and  MIL-STD-483. 

5.5.1.14  Data  Accession  list/internal 
Data  (DI-A-0327).  The  data  accession 
list  Is  an  i ndex  of  contractor  data 
which  is  available  upon  request.  It 
identifies  all  contractor  internal  data 
which  have  been  generated  by  the  contrac¬ 
tor  in  compliance  with  the  work 
described  In  the  SOW.  The  Data  Accession 
List  is  prepared  in  accordance  with 
DI-A-0327. 

5.5.1.15  TS  Documentation  Summary.  Key 
TS  computer  program  documents  have  been 
described  in  the  preceding  paragraphs. 
Each  of  these  documents  is  prepared  for 
a  specific  purpose  and  none  are  redun¬ 
dant.  The  preceding  paragraphs  cover  the 
documents  that  are  prepared  by  the  con¬ 
tractor  and  affect  computer  programs. 
Included  are  documents  prepared  specifi¬ 
cally  for  computer  programs  and  those 
prepared  for  the  entire  TS  system.  In 
some  cases,  when  computer  programs  are 
treated  as  configuration  items,  docu¬ 
ments  such  as  test  plans,  test  proce¬ 
dures  and  test  reports  will  be  prepared 
specifically  for  computer  programs. 

One  question  that  frequently  arises  is 
the  format  in  which  the  documents  are 
prepared.  The  formats  of  the  aforemen¬ 
tioned  documents  are  specified  in  the 
controlling  DIDs  with  the  exception  of 
the  Computer  Program  Product  Specifica¬ 
tion  and  the  Training  Equipment  Computer 
Program  Documentation.  Frequently  the 
contractor  will  suggest  that  this  Inter- 


nal  documentation  nearly  satisfies  the 
required  DIDs  but  is  in  a  different  for¬ 
mat. 

The  contractor  may  propose  changes  to 
the  DIDs  as  part  of  his  proposal.  These 
changes  should  be  evaluated  as  to 
whether  they  satisfy  the  document 
requirements.  Cost  alone  should  not  be 
the  deciding  factor,  but  rather  whether 
the  required  data  is  present  and  in  a 
form  that  is  convenient  to  use. 
Contractor  formats  are  different  and 
therefore  may  be  difficult  to  evaluate. 
Standard  formats  are  different  and  there¬ 
fore  may  be  difficult  to  evaluate.  Stan¬ 
dard  formats  are  preferable,  but  contrac¬ 
tor  formats  should  be  accepted  when  con¬ 
ditions  of  contractor  formats  and  data 
content  warrant  this.  Good  ideas  and 
better  methods  of  presenting  data  should 
be  used  to  update  the  present  set  of 
DIDs  and  in  the  long  run  to  minimize  con¬ 
tractor  unique  formats. 

Another  question  Is  the  effect  of 
TDSP  techniques  on  computer  program  docu¬ 
mentation.  Use  of  TDSP  may  affect  the 
product  specification,  test  plans  and 
procedures,  the  CPDP  and  the  training 
equipment  computer  program  documenta¬ 
tion.  These  DIDs  have  no  provisions  for 
TDSP  techniques  and  must  be  updated. 
Program  logic  flows  may  be  expressed  in 
a  PDL  rather  than  flow  charts.  Top  down 
integration  of  computer  programs  would 
affect  test  plans  and  configuration 
management  techniques.  The  effects  are 
still  controversial  but  appear  to  be 
beneficial  as  more  visibility  is 
generally  provided.  DIDs  should  be 
tailored  to  take  advantage  of  these 
techniques. 

5.5.1.16  TS  CDRL  Checklist.  Table 
5.5-1  is  a  checklist  for  the  selection 
of  a  CDRL  for  TS  computer  program  docu¬ 
mentation. 

5.5.2  ATE  Documentation 

The  following  subsections  provide 
descriptions  of  key  documents  for  ATE 


computer  programs,  a  summary  of  the 
required  documentation  and  a  checklist. 
Table  5.5-2,  for  selecting  the  CDRL  for 
ATE  computer  programs.  The  Support  Equip¬ 
ment  Plan  (SEP)  and  SERD  are  early  docu¬ 
ments  in  leading  to  the  ATE  contract 
supplement.  While  they  are  not  directly 
applicable  to  computer  program  documenta¬ 
tion  they  are  included  here  as  vital 
documents  for  the  weapon  system  CDRL. 
Identification  of  key  contractor- 
prepared  documents  for  the  acquisition 
of  ATE  computer  programs  is  based  on 
DODD  5000.29,  Management  of  Computer 
Resources  for  Major  Defense  Systems, 
"Defense  systems  computer  resources, 
including  both  computer  hardware  and  com¬ 
puter  software  will  be  specified  and 
treated  as  configuration  items."  Satisfy¬ 
ing  this  requirement  implies  satisfac¬ 
tion  of  the  need  for  documentation  speci¬ 
fied  in  paragraph  3.1.  These  needs  are 
satisfied  through  the  contractor- 
prepared  documents  listed  below.  This 
list  is  based  on  current  acquisition 
practices  and  contractor  experience. 

Support  Equipment  Plan 
Support  Equipment  Recommendation  Data 
Computer  Program  Development  Plan 
Computer  Program  Development  Specifi¬ 
cation 

Test  Requirements  Document 
Interface  Design  Description 
Computer  Program  Product  Specifica¬ 
tion 

Test  Plans/Procedures 
Test  Reports 

Computer  Programming  Manual 
User's  Manual 

Computer  Software  Maintenance  Manual 
Version  Description  Document 
Configuration  Management  Index 
Configuration  Index 
Change  Status  Report 
Engineering  Change  Proposal 
Specifiation  Change  Notice 
Data  Accession  List 

ATE  computer  programs  are  usually 
separated  into  three  categories.  These 
are:  test  software,  control  software  and 
support  software.  A  fourth  category. 
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Table  5.5-2.  Automatic  Test  Equipment  CDRL  Checklist  (Sheet  1  of  4) 


1.  Does  the  weapon  system  CDRL  specify  the  following  documents  or  their 
equivalent? 


2. 


DI-A-6102 

Support  Equipment  Plan 

DI-S-6176 

Support  Equipment  Recommendation  Data 

DI-T-3734 

Test  Requirements  Document 

DI-E-3108 

Configuration  Management  Plan 

DI-E-3122 

Configuration  Index 

DI-E-3123 

Change  Status  List 

DI-E-3128 

Engineering  Change  Proposal 

DI-E-3134 

Specification  Change  Notice 

DI-E-3027 

Data  Accession  List/Internal  Data 

Does  the  weapon  system  CDRL  or  CCP  CDRL  specify  the  following  ATE  computer 
program  related  documents  for  each  CPCI? 


3. 

4. 

5. 


UDI-S-3911/ASD  Computer  Program  Development  Plan 

DI-E-3119A  Computer  Program  Development  Specification 

Interface  Design  Description 


DI-E-3120 

DI-T-3703 

DI-T-3717 

DI-M-3411 

DI-M-3410 

DI-E-3121 


Computer  Program  Product  Specification 

Category  I  Test  Plans/Procedures  (Computer  Programs) 

Category  I  Test  Report  (Computer  Programs) 

Computer  Programming  Manual 
Computer  Program  User's  Manual 
Version  Description  Document 


Have  DID's  been  tailored  to  satisfy  all  requirements  for  the  specific 
application? 

Has  each  DID  been  examined  to  ensure  It  satisfies  the  requirements  of  the 
specific  application? 

Have  appropriate  DID's  been  modified  for  use  of  top  down  structured 
programming  techniques,  including  use  of  a  program  design  language? 
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Table  5.5-2.  Automatic  Test  Equipment  CDRL  Checklist  (Sheet  2 of  4) 


6.  Have  all  document  users  been  Identified?  Have  they  been  consulted? 

7.  Are  all  documentation  requirements  known? 

8.  Has  time  been  established  for  review  and  delivery  for  each  document?  Are 
schedules  related  to  specific  milestone  events  such  as  PDR,  COR,  etc? 

9.  Do  the  DID's  for  the  following  documents  contain  provisions  for  key  Items 
as  shown  below? 

Computer  Program  Development  Plan 

a.  Contractor  error  detection,  correction  and  control  procedures 

b.  Plan  for  developing  computer  programs  and  supporting  resources 

c.  Identification  of  products  to  be  delivered 

d.  Description  of  computer  program  development  organization 

e.  Control  of  design  changes 

f.  Configuration  management  techniques  during  computer  program 
devel opment 

Computer  Program  Development  Specification 

a.  Explicit  definition  of  what  the  program  shall  do 

b.  Performance  parameters  to  define  how  well  it  shall  perform 

c.  Validation  requirements 

d.  Interfaces  with  vendor  supplied  programs  for  changes  to  control  and 
support  software 

e.  Each  requirement  singularly  expressed  and  identified 

f.  References  to  TRD  for  test  software 

g.  ATLAS  statement  preparation  requirements  for  test  software 

h.  Test  software  Interfaces  with  support  and  control  software  CPCI's 
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Table  5.5-2.  Automatic  Test  Equipment  CDRL  Checklist  (Sheet  3  of  4) 


Test  Requirements  Document 

a.  Tailored  for  compatibility  with  the  test  software  development 
specification 

b.  Designation  of  parts  to  be  written  in  ATLAS  if  any 
Interface  Design  Description 

a.  Interfaces  specified  separately  or  adequately  described  In  the 
product  specification  DID 

b.  External  interface  descriptions  including: 

Data  Formats 

Methodology  for  passing  and  receiving  data 
Frequency 

c.  Internal  interface  descriptions  including: 

Data  base  structure 

Methodology  for  information  transfer 

Data  types 

Data  files 

Size 

Set/used  information 
Product  Specification 

a.  Complete  description  of  computer  programs  including: 

Descriptive  narrative 
Logic  flows 
Program  listings 

b.  Computer  timing  and  sizing  estimates 

c.  Top  down  structured  programming  techniques  where  applicable 

d.  ATLAS  statements  where  applicable 
Test  Plans/Procedures 


a.  Plans  for  all  levels  of  computer  program  testing 

b.  Delivery  of  as-run  test  procedures 

c.  Top  down  integration  techniques  where  applicable 
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Table  5.5-2  Automatic  Tact  Cquipmant  CDRL  Checklist  ( Sheet  4  of  4! 


Computer  Programming  Manual 

a.  Separate  manuals  for  each  language/computer  combination 
User  Manual 

a.  Program  design  conventions  and  philosophy 

b.  Operating  Instructions 

Initiating  operation 
Maintaining  operation 
Terminating  operation 
Restart 

c.  System  generation  procedures 

d.  Top  down  structured  programming  techniques  if  applicable 

e.  Program  design  language  description  if  applicable 
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self-test  software  is  sometimes  con¬ 
sidered  separately,  but  is  included  in 
the  test  software  category  for  the  pur¬ 
poses  of  this  guidebook.  Each  category 
may  be  handled  separately  with  each  type 
identified  as  a  CPCI.  Each  type  has  its 
own  characteristics;  where  applicable, 
each  type  will  be  described  separately. 

5. 5. 2.1  Support  Equipment  Plan 
(DI-A-6102).  The  SEP  is  a  contractor  pre¬ 
pared  document  resulting  from  an  ORLA. 
The  ORLA  is  an  iterative  decision  pro¬ 
cess  conducted  by  the  contractor  through¬ 
out  the  validation,  development  and  pro¬ 
duction  phases  of  a  weapon  system  life 
cycle.  The  analysis  considers  all  mainte¬ 
nance  factors  and  determines  the  optimum 
level  of  repair  for  all  weapon  system 
configuration  items,  i.e. ,  organiza¬ 
tional,  intermediate  or  depot.  It  identi¬ 
fies  the  repair  locations,  the  extent  of 
maintenance  permitted  and  the  resources 
necessary  to  support  the  repair  process. 
The  ORLA  will  provide  the  justification 
of  any  contractor  recommendation  for  sup¬ 
port  resources. 

The  SEP  is  prepared  using  data  resulting 
from  an  ORLA.  The  plan  describes  how  the 
contractor  will  develop  the  support 
equipment  resources.  Upon  approval  of 
the  SEP  by  the  Air  Force  the  contractor 
begins  preparation  of  the  SERD.  The  SEP 
is  prepared  in  accordance  with 
DI-A-6102.  Figure  3.3-1  shows  the  SEP  in 
relation  to  the  TRD  and  SERD.  It  is  pre¬ 
pared  during  the  full  scale  development 
phase  prior  to  approval  or  in  support  of 
a  contract  change  for  ATE.  The  document 
is  not  prepared  specifically  for  com¬ 
puter  programs,  but  the  ATE  engineer 
should  ensure  that  it  is  specified  in 
the  weapon  system's  CDRL. 

5. 5. 2. 2  Support  Equipment  Recommenda¬ 
tion  Data  (DI-S-6176). 

The  SERD  provides  the  contractors' 
recommendation  for  specific  support 
equipment  to  be  used  for  weapon  system 
maintenance.  It  will  identify  the  con¬ 
tractor's  choice  for  ATE  including  the 


computing  equipment  systems  required. 
Air  Force  concurrence  with  the  SERD  is 
required  prior  to  authorization  for  the 
contractor  to  proceed  with  further  devel¬ 
opment  or  procurement  of  ATE.  The  SERD 
will  identify  ATE  hardware  and  software 
and  other  types  of  support  equipment. 
The  SERD  is  prepared  in  accordance  with 
DI-S-6176.  It  is  prepared  during  the 
full  scale  development  phase  as  shown  in 
Figure  3.3-1.  The  SERD  is  the  result  of 
extensive  analyses  conducted  by  the  con¬ 
tractor  with  close  Air  Force  monitoring 
and  involvement.  The  SERD  is  a  vital 
part  of  the  ATE  requirements  definition 
and  must  be  included  in  the  weapon  sys¬ 
tem  CDRL. 

5. 5. 2. 3  Computer  Program  Development 
Plan  (UDI-S-391 1/ASD) •  The  CPDP  is  one 
of  the  major  computer  resource  planning 
documents.  AFR  800-14  Volume  II  requires 
a  CPDP  for  the  acquisition  of  computer 
programs.  It  is  normally  prepared  by  a 
weapon  system  contractor  for  the  imple¬ 
menting  agency  after  a  contract  change 
for  ATE  has  been  negotiated.  If  ATE  is 
acquired  under  a  separate  contract  the 
CPDP  is  prepared  as  part  of  the  contrac¬ 
tor  proposal.  The  CPDP  applies  to  all 
phases  of  the  computer  program  develop¬ 
ment  cycle. 

It  defines  the  contractor's  overall  plan 
for  developing  computer  programs  and 
necessary  supporting  resources.  The  plan 
includes  identification  of  the  program 
products  to  be  delivered,  the  schedule 
for  each  and  related  documentation.  It 
includes  a  description  of  the  develop¬ 
ment  organization;  responsibilities  for 
design,  implementation,  testing  and  inte¬ 
gration;  required  hardware  and  facili¬ 
ties  and  procedures  for  managing  and  con¬ 
trolling  all  aspects  of  development.  The 
CPDP  should  cover  all  three  types  of  ATE 
software  and  show  the  CPCI  identifica¬ 
tions.  The  development  may  be  signifi¬ 
cantly  of  ATE  software  and  show  the  CPCI 
Identifications.  The  development  may  be 
significantly  different  for  each  type. 
For  example,  all  support  software  may  be 
provided  by  a  computer  vendor;  control 
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software  may  be  obtained  from  the  com¬ 
puter  vendor  but  require  additions  or 
modifications  to  be  made  either  by  the 
vendor  or  by  the  contractor's  software 
organizations;  test  software  may  be 
coded  and  validation  by  a  test  or  design 
organization.  In  any  case,  the  entire 
process,  the  organizational  tasks  and 
responsibilities  are  specified. 

The  CPDP  should  be  used  by  the  contrac¬ 
tor  to  describe  his  procedures  for  con¬ 
trolling  design  changes  prior  to 
establishment  of  configuration  manage¬ 
ment  baselines.  During  module  and  Inte¬ 
gration  testing,  the  CPDP  should  address 
the  reporting  and  management  of  dis¬ 
crepancies  discovered  in  testing, 
responsibilities  for  failure  analysis 
and  correction,  retesting  and  the  con¬ 
trol  of  both  source  and  object  code.  In 
addition,  the  CPDP  should  describe  the 
contractor's  approach  to  performance 
estimation  and  refinement  of  the  esti¬ 
mates  In  terms  of  responsibilities, 
resources  allocation  and  relationships 
to  the  development  schedule. 

Since  the  normal  method  for  acquiring 
ATE  is  by  augmenting  to  a  weapon  systems 
contract  by  a  contract  change,  the  CPDP 
is  included  in  the  CDRL  as  a  separate 
document  devoted  to  ATE  computer  pro¬ 
grams.  It  Is  commonly  placed  on- 
contract,  requiring  the  contractor  to 
observe  the  procedures,  control  and 
method  defined  therein.  There  are  some 
disadvantages  to  this  practice  such  as 
being  contractually  committed  to  a  given 
organizational  structure  or  to  schedules 
which  may  prove  unrealistic.  These  dis¬ 
advantages  not  only  affect  the  contrac¬ 
tor  but  the  Air  Force  as  well,  requiring 
negotiating  of  any  changes  that  may 
occur. 

There  are  several  different  DIDs 
describing  CPDPs,  four  of  which  are 
Identified  below: 

a.  DI-A-5239 

b.  DI-S-3591A/M 


c.  UDI-S-3911/ASD 

d.  UDI-E-695/ESD 

These  DIDs  are  similar,  but  all  differ 
In  some  aspect  from  the  content  speci¬ 
fied  In  AFR  800-14  Volume  II.  Each  CPDP 
must  be  designed  to  fit  a  specific  appli¬ 
cation;  the  corresponding  DID  must  be 
specifically  tailored  to  that  applica¬ 
tion.  If  the  CPDP  Is  to  be  placed  on- 
contract,  great  care  must  be  taken  to 
tailor  the  DID  In  such  a  way  that  It 
satisfies  the  objectives  of  committing 
the  contractor  to  a  specified  develop¬ 
ment  approach,  but  does  not  contain 
unnecessary  constraints.  The  CPDP  must 
be  able  to  accommodate  changes  In 
requirements  during  the  development 
period.  Thus,  the  contractor  should  be 
directed  (In  the  SOW)  to  update  the  CPDP 
at  specified,  appropriate  Intervals  such 
as  the  preliminary  design  review  and 
critical  design  review. 

5. 5. 2. 4  Computer  Program  Development 
Specification  (DI-E-3119A).  The  purpose 
of  a  Computer  Program  Development  Speci¬ 
fication  is  to  provide  the  functional, 
performance  and  quality  assurance 
requirements  for  a  CPCI.  It  Is  prepared 
by  the  ATE  contractor  during  the 
analysis  phase  of  the  ATE  computer  pro¬ 
gram  development  cycle,  and  Is  one  of 
the  most  significant  products  of  that 
phase.  It  should  be  completed  and 
approved  by  the  CPCI  PDR.  Upon  approval 
by  the  Air  Force,  the  development  speci¬ 
fication  Is  placed  under  Class  I  control 
and  becomes  the  allocated  baseline.  All 
computer  programs  are  required  by  AFR 
800-14  to  be  developed  as  configuration 
Items;  thus  a  development  specification 
Is  required  for  each  ATE  CPCI. 

The  purpose  of  a  development  specifica¬ 
tion  needs  to  be  reviewed  at  this  point. 
A  development  specification  Is  prepared 
primarily  as  a  two-way  agreement  between 
the  Air  Force  and  the  development  con¬ 
tractor.  It  Is  prepared  Independent  of 
the  design  approach.  It  specifies  what 
the  software  shall  do  (function),  how 
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well  It  shall  do  It  (performance)  and 
under  what  conditions  (design  con- 
stralnts).  In  addition.  It  provides 
validation  requirements  that  define  the 
scope  of  the  validation  program.  The 
specification  Is  used  as  the  functional 
and  performance  baseline  for  the  contrac¬ 
tor  In  developing  computer  programs  and 
Is  also  used  as  the  baseline  on  which 
Air  Force  acceptance  or  rejection  of  the 
computer  program  Is  based. 

A  statement  of  computer  program  require¬ 
ments,  that  have  been  approved  by  the 
Contractor  and  the  Air  Force,  Is  a  neces¬ 
sary  Instrument  for  a  clear  understand¬ 
ing  of  what  the  contractor  will  produce 
and  what  the  Air  Force  expects.  The 
three  basic  categories  of  ATE  computer 
programs  present  different  problems  In 
the  need  and  generation  of  development 
specifications.  The  form  and  substance 
of  a  development  specification  may  vary 
for  each  CPCI  depending  on  the  category 
and  the  degree  to  which  It  must  be  devel¬ 
oped.  Since  ATE  software  categories  are 
different,  they  are  often  designated  as 
separate  CPCIs.  That  implies  separate 
specifications,  separate  development 
schedules,  separate  review  schedules  and 
separate  validation  of  requirements  that 
must  all  be  coordinated  and  eventually 
"sold"  as  a  unit.  This  raise  the  possi¬ 
bility  of  an  ATE  Software  System  Specifi¬ 
cation  covering  all  CPCIs,  as  well  as 
the  Individual  CPCI  specifications. 
Since  the  CPCIs  can  and  do  have  separate 
development  schedules,  each  CPCI  would 
have  Its  own  PDR  and  CDR  schedule;  e.g. , 
normally  the  control  and  support  soft¬ 
ware  development  precedes  that  of  the 
test  software. 

5. 5. 2. 4.1  Control  and  Support  Soft¬ 

ware.  Most  of  the  time,  control  software 
and  support  software  are  purchased  from 
an  ATE  vendor  as  part  of  a  test  set  or 
separately  from  the  computer  manufac¬ 
ture.  Control  software  and  support  soft¬ 
ware  are  usually  Identified  as  separate 
CPCIs.  When  these  computer  programs  are 
purchased  "off-the-shelf",  development 
specifications  are  not  required.  Com¬ 


puter  vendor  documentation  Is  required 
for  computer  program  maintenance  and  for 
the  possibility  of  making  changes  to  the 
purchased  computer  programs.  The  egul va¬ 
lent  of  a  product  specification  (para¬ 
graph  5. 5. 2. 7)  should  be  obtained  from 
the  vendor  If  at  all  possible.  Program 
listings  and  source  code  are  almost 
Indispensable.  When  significant  addi¬ 
tions  or  changes  must  be  made  to  the 
purchased  control  or  support  software,  a 
development  specification  should  be 
written  covering  the  changes  to  be  made, 
and  the  Interfaces  required  with  the 
purchased  computer  programs  and  the  test 
equipment.  The  existing  computer  program 
(obtained  from  the  vendor)  Is  Identified 
as  an  Interface  and  the  additional  soft¬ 
ware,  treated  as  a  CPCI,  will  be 
designed,  tested,  reviewed  and  con¬ 
trolled  accordingly.  When  control  and 
support  software  are  to  be  totally  devel¬ 
oped  by  the  contractor,  a  complete  devel¬ 
opment  specification  Is  required. 

5. 5. 2. 4. 2  Test  Software.  There  Is  con¬ 
siderable  controversy  within  the  Air 
Force  and  contractors  as  to  whether  a 
development  specification  is  applicable 
for  test  software.  One  point  of  view  Is 
that  test  software  is  a  computer  program 
and  must  be  developed  as  a  CPCI;  there¬ 
fore,  a  development  specification  Is 
required  for  proper  control  of  the  devel¬ 
opment  process.  The  other  point  of  view 
is  that  test  software  Is  derived  from  a 
TRD  (paragraph  5. 5. 2. 5)  which  can  be 
most  efficiently  written  directly  In  the 
Automatic  Test  Language  For  All  Systems 
(ATLAS)  language  by  a  UUT  design  engi¬ 
neer,  thereby,  bypassing  the  need  for  a 
development  specification  or  at  least 
regarding  the  TRD  as  the  development 
specification. 

Figure  5.5-1  Illustrates  the  engineering 
process  performed  In  the  development  of 
requirements  for  test  software.  The 
Important  point  Is  the  dependency  of 
test  software  on  test  procedures  speci¬ 
fied  In  the  TRD,  ATE  test  set  and  the 
Interface  test  adapter  (ITA).  The  TRD  Is 
prepared  for  a  production  UUT  and  Is 
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Figure  5.5-1.  Key  Events  in  ATE  Software  Specification 


Independent  of  the  test  set  (including 
support  and  control  software)  and  the 

ITA.  That  Is  unless  the  test  procedure 
portion  of  the  TRD  is  written  in  the 
ATLAS  language  and  the  ATLAS  statements 
processed  directly  by  the  ATLAS  pro¬ 
cessor.  It  is  obvious  that  the  TRD 

itself  is  not  an  adequate  substitute  for 
a  development  specification  and  cannot 
provide  the  design  definition  and  con¬ 

trol  required  for  a  CPCI. 

Since  there  is  no  consensus  on  this 
point,  the  author  presents  the  following 
proposal  which  appears  to  address  both 
sides  of  the  controversy.  First,  a  devel¬ 
opment  specification  is  required  to  pro¬ 
vide  the  basis  for  design,  management 
control  reviews,  verification,  and  con¬ 

figuration  management  requirements. 
Second,  a  provision  is  made  for  writing 
the  automatic  test  procedures  in  the 
ATLAS  language  as  part  of  the  TRD. 

Figure  5.5-2  illustrates  the  source  and 
Interactions  of  the  ATE  data  that  become 
a  part  of  the  specification.  The  inter¬ 
face  section  of  the  specification 
describes  the  Interfaces  or  design  con¬ 
straints  Imposed  by  the  control  and  sup¬ 
port  software,  CPCIs,  the  test  equipment 
and  the  ITA.  These  constitute  the  pri¬ 

mary  Interfaces.  They  obviously  cannot 
be  written  until  these  contributing  ele¬ 
ments  are  defined  as  shown  in  Figure 
5.5-1. 

The  functional  and  performance  require¬ 
ments  are  mostly  made  up  of  TRD  data  pro¬ 
duced  by  the  UUT  designers.  The  auto¬ 
matic  test  procedures  defined  by  the 
TRDs  are  referenced  by  the  specification 
and  are  written  In  the  ATLAS  language  In 
accordance  with  requirements  specified 
In  the  Design  Requirements  section  of 
the  development  specification  (or 
Special  Requirements  sections  In  the 
MIL-STD-483  format).  The  referenced 
procedures  become  part  of  the  specifica¬ 
tion.  The  remainder  of  the  functional 
and  performance  section  consists  of 
requirements  that  are  applicable  to  the 
framework  by  which  the  indivdual  auto¬ 


matic  test  procedures  are  combined  to 
produce  an  Integral  test  software  CPCI. 

The  Design  Requirements  section  consists 
of  the  functional /performance  require¬ 
ments  that  affect  the  design  of  the  test 
programs.  One  such  requirement  would  be 
the  specific  ATLAS  version  to  be  used. 
Since  the  TRDs  contain  may  Indivdual 
test  procedures  related  only  to  the  UUT, 
this  section  provides  specific  require¬ 
ments  for  the  applicable  portion  of  the 
TRD  in  ATLAS.  The  purpose  of  this 
requirement  is  to  assure  that  the  ATLAS 
code  produced  from  the  TRD  is  compatible 
with  the  remainder  of  the  ATE  computer 
program  systems. 

The  Quality  Assurance  section  consists 
of  the  requirements  to  verify  that  the 
delivered  computer  programs  perform  in 
accordance  with  the  requirements  in  the 
preceding  sections. 

The  description  of  the  development  speci¬ 
fication  Implies  a  schedule  sequence 
similar  to  that  shown  in  Figure  5.5-3. 
The  figure  shows  the  definition  of  the 
ATE  equipment,  the  ITAs  and  the  control 
and  support  software  all  precede  the 
development  specification.  The  Test  soft¬ 
ware  CPCI  PDR  is  conducted  following  com¬ 
pletion  of  the  development  specifica¬ 
tion.  The  TRDs  can  be  written  (in  ATLAS) 
anytime  after  the  design  requirements 
are  established. 

There  are  no  DIDs  designed  specifically 
for  the  different  categories  of  ATE  soft¬ 
ware.  DI-E-3119A  is  the  controlling  DID 
for  Computer  Program  Development  Specifi¬ 
cations.  There  is  a  need  for  DIDs  that 
specifically  address  these  types  of  soft¬ 
ware.  It  Is  therefore  necessary  to 
tailor  the  current  controlling  DID  for 
the  specific  type;  e.g. ,  the  test  soft¬ 
ware  DIO  could  be  tailored  to  fit  the 
development  specification  described 
above. 

5. 5. 2. 5  Test  Requirements  Document 
(DI-T-3734).  The  TRD  defines  the  testing 
required  to  validate  the  unit  to  be 


Figure  5.5-3.  Test  Software  Development 
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tested.  It  Is  usually  prepared  by  a  con¬ 
tractor  logistics  engineer  and  submitted 
to  AFLC  for  approval.  The  UUT  designer 
generates  the  required  test  procedures 
that  are  Included  In  the  TRD.  These  test 
procedures  are  prepared  In  advance  of 
the  TRD  and  are  the  basis  for  the  test 
software.  The  design  may  be  from  the 
prime  contractor's  design  organization 
or  may  be  a  subcontractor.  If  the  TRD  Is 
to  be  prepared  by  a  subcontractor,  the 
prime  contractor  should  provide  explicit 
Instructions  for  Its  preparation  to 
ensure  that  It  conforms  to  the  require¬ 
ments  specified  in  the  Computer  Program 
Development  Specification. 

As  shown  In  Figure  5.5-1,  some  TRD  data 
(based  on  the  UUT  design)  is  available 
prior  to  the  ATE  procurement.  These  data 
support  the  analyses  that  are  performed 
to  define  the  support  equipment  and  thus 
the  ATE.  The  test  procedures  to  be 
included  in  the  TRD  are  finalized  after 
the  factory  acceptance  test  procedures 
for  the  UUT.  The  test  procedure  to  be 
used  for  automatic  testing  may  be 
written  directly  in  ATLAS  if  the 
requirements  for  preparation  In  ATLAS 
have  been  previously  specified  in  the 
development  specification.  If  the 
automatic  test  procedure  portion  is  not 
written  in  ATLAS,  it  must  be  converted 
at  a  later  time.  Direct  preparation  of 
the  TRD  In  ATLAS  is  controversial;  some 
advocate  It,  believing  it  to  be  more 
efficient;  others  feel  that  not  enough 
control  is  obtained.  Writing  the  test 
procedure  directly  in  ATLAS  can  be  done 
efficiently  If  appropriate  planning  and 
preparation  is  done  before  hand. 

The  existing  DID,  DI-T-3734,  does  not 
describe  adequately  a  TRD  to  be  used  for 
ATE  In  the  context  described  in  this  and 
the  preceding  paragraph.  The  DID  refer¬ 
ences  MIL-STD-1519  which  was  written 
before  ATE  was  In  wide  use.  The  TRD  Is 
now  used  In  a  wider  context  than  origi¬ 
nally  Intended.  It  Is  therefore  neces¬ 
sary  to  tailor  the  DID  such  that  the 
Information  can  be  used  to  support  the 
Computer  Program  Development  Specifica¬ 


tion.  The  DID  should  specify  whether  the 
test  procedure  Is  to  be  written  In  ATLAS 
or  not.  It  should  specify  the  form  In 
which  the  TRD  should  be  written  to  sup¬ 
port  the  development  specification.  In 
most  cases  the  contractor  should  be  con¬ 
sulted  since  he  will  prepare  the  develop¬ 
ment  specification. 

If  the  UUT  Is  designed  and  built  by  a 
subcontractor,  the  TRD  should  be  pre¬ 
pared  by  the  subcontractor  in  accordance 
with  explicit  direction  from  the  prime 
contractor.  These  Instructions  should  be 
a  part  of  the  contractual  arrangement 
with  the  subcontractor. 

5. 5. 2. 6  Interface  Design  Description 

(No  Applicable  DID).  Interface  design  is 
of  more  than  passing  interest  in  com¬ 
puter  program  development.  When  a  pro¬ 
ject  requires  more  than  one  or  two  pro¬ 
grammers,  exact  Interface  definitions 
between  programs  and  between  program  com¬ 
ponents  become  of  prime  Importance  in 
communicating  between  programmers. 
Interface  design  is  a  computer  program 
system  engineering  task.  It  Is  the  frame¬ 
work  within  which  the  various  programs 
and  components  exchange  information. 
Data  base  design  is  integral  to  inter¬ 
face  definition.  Interfaces  can  be  con¬ 
sidered  at  two  levels:  (1)  Interfaces 
between  a  computer  program  and  external 
devices  through  I/O  channels  (primarily 
control  software),  and  (2)  Interfaces 
betwen  computer  programs  and  computer 
program  components.  Both  are  of  vital 
Importance  during  development  and  main¬ 
tenance  phases.  Internal  Interfaces 
affect  only  the  computer  program 
designer  and  maintenance  personnel, 
external  interfaces  affect  other  organi¬ 
zations. 

Experience  has  shown  that  Interface 
definitions  tend  to  be  overlooked  when 
they  are  emphasized  in  the  development 
process.  This  lack  of  definition  leads 
to  confusion  among  the  programmers  and 
between  programmers  and  other  system 
designers.  When  given  proper  emphasis  by 
preparing  separate  interface  documents. 
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a  better  definition  has  resulted;  It  has 
been  easier  to  review,  and  Interface 
Information  has  been  easire  to  locate 
and  use  for  troubleshooting  and  main¬ 
tenance. 

Interface  descriptions  for  test  software 
are  controversial.  Some  feel  that  the 
subject  is  adequately  covered  in  the 
TRD.  However,  if  test  software  is  to  be 
the  result  of  a  preconceived  design  and 
not  just  a  collection  of  TRDs  it  will  be 
specified  in  a  Computer  Program  Develop¬ 
ment  Specification  and  there  will  be 
interface  descriptions  that  are 
pertinent  to  the  computer  program  being 
developed,  (see  Figure  5.5-2  and  para¬ 
graph  5. 5. 2. 4. 2).  These  interfaces  are 
not  covered  in  the  TRD. 

The  application  to  control  and  support 
software  is  more  traditional.  The  appli¬ 
cation  depends  on  how  much  new  design  is 
required.  Interface  documentation  may 
not  be  required  if  all  control  software 
and  support  software  are  off-the-shelf, 
and  the  vendor’s  documentation  is  ade¬ 
quate.  If  changes  are  planned  or  antici¬ 
pated,  an  interface  description  must  be 
provided. 

The  interface  design  description  docu¬ 
ment  includes  detailed  description  of 
all  external  interfaces.  The  exact  for¬ 
mat,  frequency  and  methodology  for 
passing  and  receiving  data  are  included. 
Internal  interface  descriptions  include 
the  data  base  structure,  methodology  for 
passing  and  receiving  data,  detailed 
descriptions  of  files,  and  of  the 
indivdual  data  elements.  Included  are 
data  types;  e.g. ,  array,  files,  items, 
etc.;  data  characteristics;  e.g.,  float¬ 
ing  point,  integer,  binary,  BDC,  etc.; 
data  identifiers;  size;  i.e.,  number  of 
bits,  bytes,  words;  identification  of 
programs  that  set  or  use  the  data  items; 
and  a  description  of  each  data  item.  The 
organization  of  data  items  into  files  or 
other  data  structures  is  also  shown  with 
names,  dimensions  and  other  distinguish¬ 
ing  characteristics  of  the  data  base 
files. 


In  current  ATE  acquisitions  tnese  data 
are  included  as  part  of  its  product 
specification.  There  are  no  DIDs 
describing  computer  program  interface 
design  descriptions.  Therefore,  a  unique 
DID  must  be  generated  and  submitted  to 
the  Command  Data  Management  Office  for 
approval  by  the  Contractor  Data  Manage¬ 
ment  Review  Board.  It  must  be  approved 
before  it  can  be  placed  on  contract.  In 
the  meantime,  the  sections  in  the  pro¬ 
duct  specifications  can  be  strengthened 
by  tailoring  the  existing  DIDs, 
DI-E-3120  or  by  locating  a  DID  that  pri¬ 
marily  addresses  interface  definitions 
and  tailor  it  for  ATE  applications, 
e.g.,  B-l  Avionics  software  documenta¬ 
tion  contains  a  separate  document  for 
computer  program  interfaces.  A  unique 
DID  or  expanded  section  of  the  product 
specification  could  be  developed,  based 
on  the  material  included  in  the  B-l 
Avionics  example. 

The  interface  design  description  docu¬ 
ment  is  prepared  by  the  contractor  in 
parallel  with  the  product  specification. 
A  preliminary  draft  is  prepared  at  the 
CPCI  PDR;  and  a  complete  draft  at  the 
CPCI  CDR.  Separate  PDRs  and  CDRs  may  be 
conducted  for  each  CPCI  due  to  different 
development  schedule  requirements. 

5. 5. 2. 7  Computer  Program  Product  Speci¬ 
fication  (DI-E-3120).  The  product,  speci¬ 
fication  includes  a  complete  description 
of  ATE  computer  programs  including  logic 
flows  and  program  listings  supported  by 
appropriate  narrative.  A  preliminary 
draft  is  prepared  for  the  CPCI  PDR,  a 
complete  draft  is  prepared  at  CPCI  CDR 
and  it  is  delivered  at  the  PCA.  Follow¬ 
ing  acceptance,  the  configuration  man¬ 
agement  product  baselie  is  established. 
Changes  from  this  time  on  require  Air 
Force  approval.  Prior  to  delivery  to  the 
Air  Force,  the  product  specification  pro¬ 
vides  a  means  for  baseline  control, 
internal  to  the  contractor's  organiza¬ 
tion,  for  design  reviews  and  for  informa¬ 
tion  exchanges  among  programmers. 

The  Computer  Program  Product  Specifica- 
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tion  is  prepared  in  accordance  with 
DI-E-3120.  This  DID  references 
MIL-STD-483  for  format  and  contents.  A 
separate  DID  should  be  prepared  for  each 
of  the  ATE  computer  program  categories 
since  they  have  unique  requirements.  The 
DIDs  should  be  tailored  for  the  unique 
characteristics  of  ATE  computer  pro¬ 
grams.  Interface  design  descriptions  are 
addressed;  but,  as  discussed  in  para¬ 
graph  5. 5. 2. 6,  should  either  be  expanded 
or  removed  and  included  in  a  separate 
document.  ATE  computer  programs  lend 
themselves  to  a  different  emphasis 
depending  on  which  of  the  three  types 
they  are.  Test  software  is  produced  by 
converting  test  requirements  from  the 
TRD/devel opment  specification  into  the 
ATLAS  language.  The  product  specifica¬ 
tion  consists  primarily  of  the  ATLAS 
listings  with  appropriate  annotations; 
and  the  design  of  test  software  CPCI, 
possibly  expressed  with  logic  flows  and 
supporting  narrative;  and  the  quality 
assurance  section  describing  the  methods 
used  for  validation.  Interface  descrip¬ 
tions  will  be  included  if  a  separate 
document  is  not  provided.  Support  and 
control  software  consists  primarily  of 
vendor  documentation  describing  the  com¬ 
puter  programs  they  provided.  Logic 
flows  and  listings  are  necessary  for 
efficient  maintenance;  they  should  be 
obtained  if  possible.  Changes  required 
for  the  support  and  control  software 
must  be  fully  documented  including  devel¬ 
opment  specification,  interface  descrip¬ 
tion  and  a  product  specification.  Logic 
flows,  annotated  listings  and  narrative 
are  required  in  addition  to  quality 
assurance  requirements. 

ATE  product  specifications  must  be  kept 
up-to-date  if  they  are  to  be  of  value  to 
either  the  contractor,  the  developing 
agency  or  the  maintenance  agency.  The 
product  specification  presents  a  com¬ 
plete  design.  Computer  code  Is  generated 
In  accordance  with  the  design.  Changes 
are  inevitable  and  are  expected.  The  con¬ 
tractor's  approach  to  handling  changes 
and  keeping  the  draft  product  specifica¬ 
tion  current  are  described  in  the  CPDP. 


Logic  flows  and  computer  timing  and 
sizing  estimates  in  particular  remain 
fluid  and  require  effort  to  keep 
current.  After  delivery  to  the  Air 
Force,  it  is  even  more  difficult  to  main¬ 
tain  currency  due  to  the  length  of  time 
to  approve  changes  through  change  board 
action. 

The  product  specification  consists  of 
three  primary  parts:  logic  flow  dia¬ 
grams,  supporting  narrative  and  the  pro¬ 
gram  source  listings.  Care  should  be 
taken  to  assure  that  these  parts  can  be 
easily  correlated  by  clearly  annotating 
these  parts.  The  program  listing  repre¬ 
sents  the  executable  computer  program 
code;  and  the  logic  flow  and  the  narra¬ 
tive  description  represents  the  design. 
Top  down  structured  techniques  may  be 
used  in  some  cases,  when  this  occurs  the 
logic  flows  may  be  expressed  in  a  POL  in 
place  of  the  traditional  flow  charts. 
The  product  specification  DID  should  be 
tailored  to  provide  this  possibility. 

5. 5. 2. 8  Computer  Program  Test  Plans/ 

Procedures  (DI-T-3703).  Test  plans  are 
prepared  for  each  CPCI  to  show  the  faci¬ 
lities,  methods,  personnel  and  schedules 
required  for  validating  ATE  computer  pro¬ 
grams  from  early  programmer  tests  to 
final  acceptance.  The  test  plan  Includes 
provisions  for  all  three  types  of  ATE 
Computer  programs.  Test  plans  may  be  pre¬ 
pared  as  a  single  document  or  separate 
documents  for  each  CPCI  describing  how 
the  different  CPCIs  will  be  tested  and 
integrated.  If  schedules  are  compatible 
a  single  ATE  test  plan  is  preferable. 

Test  plans  and  test  procedures  are  pre¬ 
pared  by  the  contractor,  describing  in 
detail  the  planning,  equipment  required, 
test  schedules  and  detailed  step-by-step 
procedures  for  validation  of  ATE  soft¬ 
ware  In  accordance  with  DI-T-3703.  Test 
plans  are  prepared  in  accordance  with 
section  4  of  the  development  specifica¬ 
tion.  In  the  past,  validation  of  test 
software  has  lacked  formality.  It  Is 
essential  that  test  software  be  quali¬ 
fied  in  accordance  with  the  development 
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specification  to  provide  necessary 
quality  assurance  provisions.  Test  plans 
for  test  software  will  describe  the 
methods  used  for  testing,  the  equipment 
required  and  test  schedules.  Care  is 
required  for  coordinating  the  avail¬ 
ability  of  hardware  and  software  facili¬ 
ties.  Test  plans  for  control  and  support 
software  will  address  how  the  specified 
additions  and  changes  will  be  tested. 
Preliminary  draft  of  the  test  plans 
should  be  available  at  the  CPCI  PDRs; 
the  complete  drafts  are  reviewed  at  the 
CPCI  CDRs.  Test  plans  should  show  the 
entire  test  plan  leading  to  acceptance 
of  the  ATE  software.  This  includes  pro¬ 
grammer  level  tests,  computer  program 
integration  tests,  and  validation  tests. 

Test  procedures  are  prepared  for  each 
individual  qualification  test  and  will 
specify  the  test  sequence,  including 
identification  of  the  equipment  con¬ 
figuration,  computer  program  configura¬ 
tion,  equipment  setup,  step-by-step  pro¬ 
cedures  and  expected  results.  The  con¬ 
tractor  may  provide  internal  test  proce¬ 
dures  for  lower  level  tests  leading  to 
qualification.  These  lower  level  proce¬ 
dures  are  not  listed  on  the  CDRL  but  may 
be  obtained  through  the  data  accession 
list  technique  (paragraph  5.4)  if 
desired. 

Test  plans  and  test  procedures  should  be 
prepared  in  separate  volumes.  Test  plans 
are  normally  produced  well  in  advance  of 
test  procedures.  It  may  also  be  more  con¬ 
venient  to  prepare  test  procedures  In 
separate  volumes  depending  on  the  size 
of  the  tests.  Test  procedures  for  formal 
qualification  and  acceptance  tests  must 
be  approved  by  the  Air  Force  prior  to 
actual  testing.  Changes  made  during  test¬ 
ing  must  be  recorded  In  the  test  proce¬ 
dure  to  provide  a  record  of  the  test  as 
It  was  conducted.  The  as-run  test  proce¬ 
dure  Is  delivered  at  FCA. 

5. 5. 2. 9  Test  Reports  (DI-T-3717).  Test 
reports  are  prepared  by  the  contractor 
to  describe  the  results  obtained  from 
qualification  tests.  They  are  prepared 


in  accordance  with  DI-T-3717.  They  are 
delivered  within  a  time  period  following 
the  test  completion.  The  delivery  should 
be  specified  in  block  12  of  the  CDRL 
form  1423.  The  delivery  schedule  should 
be  expressed  in  terms  related  to  comple¬ 
tion  of  a  test,  i.e.,  delivery  required 
60  days  after  completion  of  test.  The 
results  are  used  as  part  of  the  FCA  pro¬ 
cess.  Delivery  of  lower  level  test 
results  may  be  obtained  using  data  acces¬ 
sion  list  techniques  (paragraph  5.4). 

5.5.2.10  Computer  Programming  Manual 
(DI-M-3411).  The  computer  programmi ngr 
manual  provides  instructions  to  enable 
experienced  computer  programmers  to  pre¬ 
pare,  interpret  and  alter  computer  pro¬ 
grams.  These  programs  are  written  in  a 
particular  machine,  assembly,  or  com¬ 
piler  language  for  a  specific  computer. 
Compiler  languagers  will  include  FORTRAN 
for  control  and  support  software  and 
ATLAS  for  test  software..  Computer  pro¬ 
gramming  manuals  are  required  for  each 
language/computer  combination.  Vendor 
manuals  are  sufficient  if  they  are  avail¬ 
able.  If  they  are  not  available  a  manual 
must  be  prepared  in  accordance  with 
DI-M-3411.  Control  and  support  computer 
programming  manuals  are  normally  pro¬ 
vided  by  the  ATE  computer  manufacturers. 

The  ATLAS  compiler  may  be  provided  by  a 
vendor  or  may  be  developed  (either 
partially  or  wholly)  by  the  contractor. 
If  It  is  developed  or  modified  by  the 
contractor,  a  programming  manual  is 
probably  of  greater  Importance  to  main¬ 
tenance  personnel  than  to  the  software 
developers.  If  the  ATLAS  compiler  is 
developed  by  the  contractor  using  his 
own  funds,  a  problem  of  data  rights  mqy 
surface.  It  Is  Important  that  data 
rights  are  obtained  for  all  programs  and 
data  that  are  necessary  for  computer  pro¬ 
gram  maintenance.  Data  rights  are  sub¬ 
ject  to  negotiation  and  must  be  obtained 
for  all  computer  program  documentation 
that  Is  required  for  maintenance  sup¬ 
port. 


62 


5.5.2.11  User's  Manual  (DI-M-3410). 
User's  manuals  are  developed  to  provide 
computer  program  users  with  functional 
descriptions,  usage  Instructions  and 
descriptions  of  Input  data  requirements 
and  output  products  for  the  computer  pro¬ 
gram.  In  addition.  Information  for  com¬ 
puter  operations  Is  provided  In  terms  of 
procedures  for  Initiating  the  program, 
terminating  and  restarting  It  as  well  as 
descriptions  of  operator  inputs,  out¬ 
puts,  formats,  and  Interrelationships. 
System  generation  procedures  should  also 
be  addressed,  describing  them  In  detail 
to  assure  Identical  results  can  be 
obtained  from  separate  system  building 
activities. 

The  user's  manual  Is  an  output  of  the 
development  effort  and  is  prepared  by 
the  development  contractor  for  delivery 
at  PCA.  Drafts  should  be  available  prior 
to  formal  test  and  integration  to  sup¬ 
port  these  activities.  It  provides 
Instructions  for  proper  use  of  the  pro¬ 
gram  In  the  operational  and  support 
phases  as  specified  in  DI-M-3410. 

The  DID  offers  a  comprehensive  organi¬ 
zation  for  this  manual.  It  is  necessary 
to  tailor  the  DID  for  each  specific  ATE 
application.  This  may  involve  a  DID  for 
each  CPCI  or  for  each  type  of  ATE  soft¬ 
ware.  The  specific  application  is 
dependent  on  the  amount  of  development 
work  to  be  done,  usability  of  vendor 
manuals  and  the  amount  and  type  of  main¬ 
tenance  anticipated.  User  manuals 
obtained  from  vendors  will  probably  not 
meet  requirements  of  DI-M-3410,  but  may 
be  suitable  for  the  intended  user.  This 
problem  may  be  resolved  by  having  the 
manuals  reviewed  by  personnel  of  the 
eventual  user  and/or  support  organiza¬ 
tions. 

If  TDSP  techniques  are  used,  the  TDSP 
philosophy  should  be  provided.  If  a  Pol¬ 
ls  used  In  place  of  traditional  flow 
charts,  the  PDL  philosophy  should  be 
Included  to  enable  program  maintenance 


personnel  to  understand  the  specific 
technique  used. 

The  relationship  between  a  user's  manual 
and  a  Technical  Order  (T.O.)  needs  to  be 
explored  for  test  software.  Test  soft¬ 
ware  should  be  designed  to  provide  a 
structure  that  integrates  the  separate 
test  procedure  produced  from  the  TRDs. 
The  T.O.  is  a  detailed  procedure  for  per¬ 
forming  a  test  on  a  UUT.  The  user's 
manual  contains  some  applicable  material 
that  may  be  used  to  assist  in  preparing 
the  T.O.  However,  the  user's  manual  is 
prepared  to  assist  computer  program 
maintenance  and  operations.  The  two, 
T.O.  and  Users  Manual,  are  separate  and 
distinct  documents. 

5.5.2.12  Computer  Software  Maintenance 
Manual  (DI-M-5118).  The  Computer  Soft¬ 
ware  Maintenance  Manual  provides  a  main¬ 
tenance  programmer  with  Information  to 
enable  modification  or  maintenance  of 
computer  programs  delivered  under  a  con¬ 
tract.  DI-H-5070  is  normally  used  in  con¬ 
junction  with  this  item.  DI-H-5070 
requires  the  delivery  of  supporting 
computer  programs  that  may  be  used  by 
its  contractors,  as  tools  for  more 
efficient  means  of  debugging  and 
updating  this  computer  program. 
Tailoring  for  a  specific  application  is 
required. 

These  DIDs  are  shown  here  as  an  alterna¬ 
tive  consideration  to  the  User  Manual 
(paragraph  5.5.2.11).  The  maintenance 
manual  and  the  user's  manual  contain 
much  redundant  information.  In  fact  the 
maintenance  manual  contains  information 
that  Is  also  redundant  with  the  product 
specification.  It  is  doubtful  that  both 
manuals  should  be  required.  The  user's 
manual  Is  a  more  complete  document  and 
contains  a  wide  variety  of  information. 
Either  manual  may  be  used,  when  care¬ 
fully  tailored  to  the  application  at 
hand  by  combining  Information  from  all 
three  DIDs,  reducing  the  scope  or  a  com- 


63 


bi nation  of  both.  The  user's  manual  Is 
the  more  comprehensive;  therefore  It 
should  be  the  one  used. 

5.5.2.13  Version  Description  Document 
(DI-E-3121).  The  purpose  of  the  VDD  is 
to  identify  the  exact  configuration  of 
ATE  computer  programs  and  interim 
changes  to  them.  It  is  used  to  identify 
each  version,  and  accordingly, 
accompanies  each  version  of  ATE  computer 
program  and  each  release  for  interim 
change. 

The  VDD  is  prepared  by  the  ATE  contrac¬ 
tor  in  accordance  with  DI-E-3121.  The 
DID  essentially  refers  to  MIL-STD-483 
for  format  and  content.  A  VDD  is 
delivered  to  the  Air  Force  at  PCA  for 
each  CPCI  that  provides  an  exact  descrip¬ 
tion  of  the  ATE  software  configuration 
delivered  at  that  time. 

During  the  installation  and  operational 
phases  of  the  computer  program  develop¬ 
ment  cycle,  many  different  computer  pro¬ 
gram  versions  are  generated  and  used. 
Several  of  these  versions  may  be  in  use 
concurrently.  Configuration  management 
of  computer  program  versions  are 
generated  and  used.  Several  of  these 
versions  may  be  in  use  concurrently. 
Configuration  management  of  computer 
program  evolution  can  be  a  monumental 
problem  if  there  is  no  means  for  identi¬ 
fying  the  separate  versions.  Basically  a 
new  version  is  created  whenever  any 
change  is  made.  The  VDD  covers  the 
entire  CPCI  delivered,  including  vendor 
supplied  sofware.  The  VDD  provides  the 
same  function  as  a  top  level  hardware 
drawing  in  that  it  identifies  the  exact 
version  of  each  component  that  makes  up 
the  computer  program.  The  exact  design 
and  code  changes  to  a  specific  version 
of  a  computer  program  component  are 
described  in  the  product  specification 
complete  with  logic  flow  and  program 
listings;  however,  a  composite  program 
configuration  consisting  of  specific 
versions  of  program  components  may  be 
very  difficult,  if  not  impossible,  to 
identify  by  the  product  specification. 


The  VDD  should  also  identify  the  exact 
configuration  of  all  the  computer  pro¬ 
grams  required  to  generate  and  maintain 
the  deliverable  computer  programs.  This 
Includes  the  version  identifiers  of 
compilers,  assemblers,  link  editors  and 
all  other  computer  programs  used  In  the 
process. 

5.5.2.14  Configuration  Mangement  Plan 
(DI-E-3108).  The  CMP  is  prepared  by  the 
contractor  to  describe  his  assignment  of 
organizational  responsibilities  and  the 
procedures  used  in  the  accomplishment  of 
the  specific  configuraton  management 
requirements  as  stated  in  the  SOW.  The 
CMP  is  prepared  as  part  of  the  contrac¬ 
tor  weapon  system  proposal  and  may 
become  contractual  after  contract  award 
and  approval  by  the  Air  Force.  It  is 
written  for  the  entire  weapon  system 
development,  possibly  several  years 
before  the  ATE  activity  is  seriously 
considered.  Configuration  management 
plans  are  established  at  that  time. 
Configuration  management  for  the 
operational  software  will  probably  be 
addressed.  In  all  probability  ATE  soft¬ 
ware  will  not  be  addressed,  unless  speci¬ 
fic  requirements  are  written  into  the 
DID.  The  basic  DID  is  DI-E-3108.  Proce¬ 
dures  unique  to  ATE  computer  programs 
should  be  written  in  the  ATE  CPDP. 

5.5.2.15  Configuration  Index 
(DI-E-3122).  The  configuration  index  for 
a  computer  program  is  part  of  an  inte¬ 
grated  approach  to  configuration  manage¬ 
ment  in  accordance  with  Appendix  XIV  of 
MIL-STD-483.  It  provides  the  current 
status  for  specifications  and  selected 
additional  documents  (such  as  test 
plans/procedures,  test  reports,  user's 
manuals  and  VDDs)  whose  content  is 
dependent  on  current  ATE  software  con¬ 
figurations. 

The  configuration  index  is  prepared  in 
accordance  with  DI-E-3122  and 
MIL-STD-483.  It  is  prepared  by  the  con¬ 
tractor  as  an  on  going  record  of  changes 
to  the  affected  documents.  Document  sta¬ 
tus  is  summarized  by  dates  of  issue. 
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document  numbers  and  titles,  ECPs,  SCNs, 
and  revision  Identification  associated 
with  each  Issue  or  document  change 
resulting  from  accomplished  changes. 

The  configuration  Index  Is  established 
as  part  of  the  prime  weapon  system  con¬ 
tract.  The  Initial  CDRL  should  be 
examined  to  assure  that  the  configura¬ 
tion  index  Is  on  the  list  and  that  the 
DID  provides  for  and  does  not  exclude, 
either  Implicitly  or  explicitly,  ATE 
computer  programs  of  all  categories.  If 
adequate  provisions  are  not  made  for  ATE 
computer  programs  at  the  time  the  ATE 
CCP  Is  being  negotiated,  a  new  DID  can 
be  added  specifically  for  ATE. 

5.5.2.16  Change  Status  List 

(DI-E-3123).  The  change  status  list  is 
supplementary  to  the  Configuration  Index 
(paragraph  5.5.2.15).  It  descrbes  the 
status  of  all  proposal  changes  to  the 
weapon  system  for  which  the  contractor 
is  responsible  and  for  which  existing 
documentation  is  listed  In  the  config¬ 
uration  index.  It  Is  prepared  by  the  con¬ 
tractor  in  accordance  with  DI-E-3123  and 
MIL-STD-483.  It  is  a  continuous  record 
of  the  status  of  proposed  and  approved 
changes. 

As  with  the  two  previous  documents,  the 
change  status  report  Is  Identified  in 
the  prime  weapon  system  contract  and 
applied  to  the  entire  weapon  system.  ATE 
engineering,  while  not  directly  Involved 
at  that  time,  should  review  the  CDRL  and 
the  referenced  DIDs  to  assure  the 
existence  of  the  change  status  report 
and  that  ATE  computer  programs  are  not 
Inadvertently  excluded.  If  ATE  computer 
programs  are  excluded  a  new  DID  can  be 
added  at  the  time  the  ATE  CCP  Is  being 
negotiated. 

5.5.2.17  Engineering  Change  Proposal 
(DI-E-3128).  The  ECP  Is  the  vehicle  used 
to  prepare,  process  and  Incorporate 
class  I  engineering  changes  to  the  appli¬ 
cable  configuration  management  baseline, 
l.e. ,  development  specification  or  pro¬ 
duct  specification.  It  lAisually  pre¬ 


pared  by  the  contractor  and  must  be 
approved  by  the  Air  Force.  It  is  pre¬ 
pared  when  a  change  Is  considered  appro¬ 
priate  and  in  accordance  with  DI-E-3128. 

5.5.2.18  Specification  Change  Notice 
(DI-E-3134).  The  SCN  Identifies  a  pro¬ 
posed  change  to  a  contractually  appli¬ 
cable  specification  and,  after  approval, 
provides  a  record  of  the  change  and  the 
associated  ECP.  The  SCN  Is  prepared  by 
the  weapon  system  contractor  whenever  an 
engineering  change  Is  proposed  and  Is 
Included  as  part  of  the  ECP  package.  The 

_-SCN  is  prepared  in  accordance  with 
DI-E-3134  and  MIL-STD-483.  The  SCN  Is 
Identified  in  the  prime  weapon  system 
CDRL  and  remains  applicable  throughout 
the  development  and  installation  phases 
and  includes  ATE  computer  programs, 
unless  specifically  excluded. 

5.5.2.19  Data  Acquisition  List/ 
Internal  Data  (DI-A-3027).  The  data 
accession  list  is  an  index  of  data  that 
is  available  upon  request.  It  identifies 
all  contractor  internal  data  which  have 
been  generated  by  the  contractor  in  com¬ 
pliance  with  the  work  effort  described 
in  the  SOW.  The  data  accession  list  Is 
prepared  in  accordance  with  DI-A-3027. 

5.5.2.20  ATE  Documentation  Summary. 
ATE  computer  programs  have  several 
unique  characteristics  which  should  be 
considered  during  selection  of  a  CDRL. 
These  are: 

a.  ATE  computer  program  activity 
occurs  late  (up  to  several  years)  in  the 
development  of  a  weapon  system. 

b.  ATE  activity  Is  not  prominent  at 
the  time  the  Initial  weapon  system  CDRL 
Is  being  developed  and  thus  may  be  over¬ 
looked. 

c.  CDRL  Items  that  are  peculiar  to 
ATE  computer  programs  must  be  negotiated 
with  the  contractor  when  the  ATE  CCP  Is 
being  developed. 


d.  A  significant  portion  of  the  Infor¬ 
mation  for  ATE  computer  program  docu¬ 
ments  may  come  for  subcontractors. 

e.  In  the  past,  ATE  test  software  has 
been  developed  as  a  T.O.  order  rather 
than  a  CPCI. 

5.5.2.20.1  ATE  Time  Lag.  The  time  lag 

between  the  weapon  system  development 
activity  and  ATE  development  activity 
results  In  several  problems  that  require 
aggressive  action  on  the  part  of  ATE 
engineering  personnel.  Since  ATE  com¬ 
puter  program  activity  Is  a  long  way  In 
the  future,  ATE  engineering  activity 
will  be  a  part-time  assignment  at  best. 
Responding  to  the  data  call  and  keeping 
current  with  the  proposed  and  final  CORL 
can  only  be  accomplished  by  initiating 
the  action  and  making  sure  that  follow¬ 
up  activity  is  not  overlooked.  Data 
items  that  are  unique  to  ATE  computer 
programs  will  not  be  Included  In  the 
Initial  CDRL;  however,  ATE  engineering 
should  examine  the  DIDs  for  those  items 
that  are  relevant  but  not  unique  to  ATE 
computer  programs.  These  documents  are 

TRDs,  CMP,  configuration  index,  change 
status  report,  ECP,  SCN,  and  the  data 
accession  list.  Some  current  programs 
are  engage  In  ATE  software  planning 
prior  to  the  weapon  system  RFP.  This 
practice  should  be  encouraged.  While 
many  specific  details  are  missing, 

coming  to  grips  with  the  relationships 
between  TRDs  and  development 

specifications  Is  certainly  possible. 

5.5.2.20.2  ATE  Computer  Program  Unique 

Documents.  The  documents  that  are  unique 
to  computer  programs  are  Identified 
during  the  preparation  of  the  ATE  CCP. 
The  list  must  be  negotiated  with  the 
existing  contractor(s).  The  contrac¬ 

tor  (s)  will  be  more  Independent  In  their 
demands  at  this  time  than  when 
negotiating  the  prime  contract.  They 
will  probably  have  different  Ideas  as  to 
which  documents  should  be  prepared  as 
well  as  the  contents  and  formats.  When 
evaluating  their  requests  (demands),  one 
must  always  keep  the  purposes  of  docu¬ 


mentation  In  sight  and  determine  whether 
or  not  these  purposes  are  satisfied. 
Inadequate  documentation  may  be  much 
more  expensive  than  too  much  documenta¬ 
tion  when  total  life  cycle  costs  are 
examined.  Contractor  formats  may  be 
acceptable  and  may  satisfy  documentation 
purposes  at  a  lower  cost.  Contractor  for¬ 
mats.  Standard  formats  are  preferable, 
but  contractor  formats  may  be  acceptable 
If  a  distinct  advantage  can  be  shown. 
Good  Ideas  and  better  methods  of  data 
presentation  should  be  used  to  update 
the  present  set  of  DIDs  and  In  the  long 
run  to  minimize  contractor  unique  for¬ 
mats. 

5.5.2.20.3  Subcontractor  Data.  Much  of 
the  data  required  for  ATE  computer  pro¬ 
gram  documentation  may  originate  with  a 
subcontractor.  Some  TRDs  may  be  prepared 
by  a  subcontractor  building  a  particular 
unit  to  be  tested,  ATE  equipment  and  com¬ 
puter  programs  may  be  provided  by  a 
vendor  or  computer  manufacturer,  some 
support  software  may  be  provided  by  yet 
another  vendor.  Control  of  subcontractor 
data  is  vital  to  the  efficient  develop¬ 
ment,  operation,  and  support  of  ATE  com¬ 
puter  programs. 

When  a  weapon  system  component  Is  manu¬ 
factured  by  a  subcontractor,  the  sub¬ 
contractor  should  be  required  (by  con¬ 
tract)  to  provide  a  TRD  or  equivalent. 
The  contents  and  format  of  the  TRD  for 
ATE  should  be  compatible  with  the  ATE 
Computer  Program  Development  Specifica¬ 
tion,  as  described  In  paragraphs  5.5. 2.4 
and  5. 5. 2. 5.  The  TRD  and  development 
specification  DIDs  should  be  specifi¬ 
cally  tailored  for  this  purpose. 

Vendor  data  for  computing  systems  and 
computer  programs  present  another  prob¬ 
lem.  First,  whether  the  data  Is  avail¬ 
able  at  all;  second  whether  It  Is  com¬ 
patible  with  the  need,  and  third, 
whether  the  vendor  will  make.  It  avail¬ 
able.  These  factors  should  all  be  con¬ 
sidered  when  selecting  the  ATE  equip¬ 
ment.  The  purchase  agreement  should 
Include  stipulation  as  to  what  docu- 
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mentation  Is  required  and  when  It  should 
be  delivered.  Data  rights  must  be 
negotiated  such  that  all  data  needed  for 
operation  and  maintenance  can  be 
obtained,  used  and  maintained. 

5.5.2.20.4  T.O.  Versus  CPCI.  Histori¬ 
cally,  ATE  test  software  has  been 
acquired  as  a  T.O.  DODD  5000.29  cur¬ 
rently  requires  that  they  be  acquired  as 
a  Cl.  This  Implies  more  documentation 
and  tighter  controls  for  the  development 
of  ATE  computer  programs.  Development  of 
test  software  Is  probably  further 
removed  from  the  computer  program  devel¬ 
opment  organization  than  other  types  of 
software.  Hardware  designers  write  test 


requirements,  test  engineers  may  convert 
the  test  requirements  to  ATLAS  state¬ 
ments  and  the  software  development 
organization  may  provide  configuration 
management  techniques.  The  diverse  parti¬ 
cipants  make  It  difficult  to  control 
and,  thus,  more  highly  susceptible  to 
abuses.  It  Is  also  for  this  reason  that 
controls  such  as  adequate  documentation, 
testing  and  configuration  management  are 
needed.  These  are  all  attributes  of 
developing  computer  programs  as  CPCIs. 
All  required  computer  program  documenta¬ 
tion  should  be  specified  In  the  CDRL. 
The  T.O.  should  be  used  as  an  appli¬ 
cation  of  the  program  such  as  the  UUT 
test  procedures  described  In  paragraph 
5.5.2.10. 
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Section  6.0  DOCUMENT  REVIEW  AND  APPROVAL 


Following  the  selection  of  a  proper  CDRL 
there  are  still  important  engineering 
functions  to  be  performed.  A  properly 
designed  computer  program  documentation 
scheme  provides  pertinent  documents  at 
key  milestones  in  the  development  pro¬ 
cess.  Drafts  of  many  of  the  documents 
are  generally  produced  long  before 
delivery  to  the  Air  Force.  The  drafts 
are  reviewed  at  such  milestones  at  PDR, 
CDR  or  prior  to  the  beginning  of  quali¬ 
fication  testing.  This  provides  the 
opportunity  for  assessing  the  develop¬ 
ment  status  of  the  computer  programs  and 
evaluation  of  the  documents  themselves 
at  an  early  date.  This  section  addresses 
the  tracking  of  the  computer  program 
development  status  through  documenta¬ 
tion,  early  evaluation  of  the  required 
documents  and  change  control  performed 
by  both  the  contractor  and  the  using  and 
support  agencies. 

6.1  COMPUTER  PROGRAM  STATUS 

Computer  program  documentation,  phased 
for  completion  at  appropriate  times  in 
the  computer  program  development  cycle, 
provides  an  excellent  means  for  deter¬ 
mining  contractor  program  status.  This 
concept  embodies  the  development  of 
computer  program  documentation  in  paral¬ 
lel  with  that  of  the  computer  program 
itself.  Each  of  the  phases  in  the  com¬ 
puter  program  development  cycle  produces 
documentation  representative  of  that 
particular  phase.  Figure  6.1-1  shows 
this  relationship  for  both  TS  and  ATE 
computer  program  development.  The  figure 
shows  delivery  of  the  document  to  the 
Air  Force  as  well  as  the  release  of 
review  drafts.  It  should  be  noted  that 
the  references  to  PDR,  CDR  and  FCA/PCA 
are  CPCI  related.  Each  CPCI  or  Cl  has  a 
schedule  with  its  specific  milestones. 
The  milestones  may  be  integrated  for  the 
several  CPCIs  but  for  the  purposes  of 
the  figure  they  refer  to  the  CPCI. 


6.1.1  Analysis  Phase 

The  emphasis  In  the  analysis  phase 
focuses  on  the  analysis  and  specifi¬ 
cation  of  computer  program  requirements, 
planning  for  the  development  process  and 
a  preliminary  design.  TS  and  ATE  acquisi¬ 
tion  programs  differ  in  startup  methods 
since  they  are  usually  different  types 
of  contracts. 

A  TS  system  is  usually  acquired  under  a 
separate  contract.  It  is  often  preceded 
by  a  competitive  study  phase.  The  study 
phase  is  part  of  the  analysis  phase  and 
results  in  a  TS  system  specification,  a 
contractor  proposal  and  a  computer  pro¬ 
gram  development  plan.  The  proposal  and 
the  CPDP  are  evaluated  and  are  used  as 
part  of  the  source  selection  process. 
After  award  of  the  contract,  the  system 
specification  and  the  contractor  pro¬ 
posal  are  placed  on  contract;  the  CPDP 
may  also  become  contractual  (see  para¬ 
graphs  5. 5. 1.2  and  5. 5. 2. 3).  The 
emphasis  then  shifts  to  preliminary 
design  and  test  planning  activities  to 
be  completed  and  reviewed  at  PDR. 

ATE  acquisitions  are  not  as  clear  cut. 
ATE  development  is  often  contracted  as 
an  extension  of  a  prime  weapon  system 
contract  and  is  dependent  on  tasks  per¬ 
formed  previously  by  different  organiza¬ 
tions  such  as  the  ORLA  and  the  prepara¬ 
tion  of  the  SERD  and  the  SEP.  The  docu¬ 
mentation  products  of  the  analysis  phase 
are  essentially  the  same  as  the  TS  a 
CPDP,  development  specification,  prelimi¬ 
nary  product  specification,  test  plans 
and  preliminary  interface  documents.  A 
significant  difference  is  that  the  devel¬ 
opment  specification  is  prepared  by  the 
contractor.  The  development  specifica¬ 
tion  is  usually  placed  under  Class  I 
change  control  after  approval  by  the  Air 
Force  and  becomes  a  contractual  docu¬ 
ment.  The  CPDP  may  also  be  placed  on 
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contract  as  described  in  paragraph 
5. 5. 2. 3.  PDRs  may  be  Held  at  different 
times  for  the  different  ATE  software 
categories  depending  on  the  schedule. 

TRDs  are  developed  as  a  part  of  the 
weapon  system  contract  and  do  not  show 
in  the  computer  program  development 
phases.  Ideally  they  would  be  developed 
in  the  analysis  phase;  but,  in  reality 
they  may  not  be  completed  until  well 
into  the  design  phase.  The  relationship 
to  the  TRD  and  the  test  software  devel¬ 
opment  specification  is  described  in 
paragraphs  5. 5. 2. 4  and  5. 5. 2. 5. 

The  PDR  represents  the  completion  of  the 
analysis  phase  for  a  CPCI  or  Cl.  The 
documentation  produced  at  that  time 
provides  excellent  material  for  review 
and  assessment  of  the  progress  made  by 
the  contractor  toward  fulfillment  of  the 
contract.  Contractor  planning  can  be 
reviewed  to  determine  whether  plans  are 
realistic  in  relation  to  the  progress 
made  to  date.  The  maturity  of  his  pre¬ 
liminary  design  and  interface  definition 
may  reveal  problem  areas  that  lay  ahead. 
His  understanding  of  the  problem  to  be 
solved  may  be  revealed  through  the  appli¬ 
cation  of  his  design  to  the  requirements 
for  the  computer  programs.  Good  docu¬ 
mentation  at  this  point  is  indicative  of 
a  well -managed  software  development. 
Poor  or  incomplete  documentation  may 
reveal  future  problem  areas  or  that  more 
time  is  needed  to  complete  this  phase 
before  preceeding  to  the  next  phase. 

6.1.2  Design  Phase 

The  emphasis  in  the  design  phase  is  to 
achieve  a  fully  developed  design  which 
can  be  translated  directly  into  computer 
code.  This  phase  culminates  in  the  CDR. 
Draft  product  specifications  and  inter¬ 
face  design  descriptions  should  be  com¬ 
pleted.  TRDs  are  complete  at  this  time 
and  available  for  review  for  ATE  test 
software.  TRDs  may  be  partially  written 
In  the  ATLAS  language.  These  documents 
can  be  used  as  baselines  for  contractor 


internal  configuration  management  prac¬ 
tices.  Test  plans  should  also  be 
completed.  Review  of  these  documents  at 
CDR  will  reveal  the  maturity  of  design 
and  the  readiness  for  beginning  the  code 
and  checkout  phase.  Again  poor  or  incom¬ 
plete  documentation  may  be  indicative  of 
poor  design  or  poor  management.  A  deci¬ 
sion  to  move  to  the  code  and  checkout 
phase  or  continue  in  the  design  phase 
can  be  made  Intelligently  based  on  the 
documentation  presented  at  the  CDR. 


The  emphasis  in  this  phase  is  producing 
computer  code  and  checking  out  indi¬ 
vidual  program  modules.  Documentation 
produced  during  this  phase  is  the 
computer  program  listing,  an  update  of 
the  product  specification  and  test 
procedures.  Air  Force  review  of  the 
listing  at  this  point  is  not  vital.  The 
program  listing  provides  a  vital  element 
in  the  identification  of  the  computer 
program  configuration.  A  parallel  effort 
being  performed  during  this  phase  is  the 
development  of  test  procedures  for 
qualification  of  the  computer  programs. 
These  procedures  should  be  prepared  by  a 
contractor  organization  that  is  inde¬ 
pendent  of  the  development  organization. 
Test  procedures  are  based  on  the  program 
design  described  in  the  product  specifi¬ 
cation  and  the  requirements  specified  in 
the  development  specification. 

Completion  of  test  procedures,  the 
program  listings  and  the  updated  product 
specifications  is  required  for  contin¬ 
uation  Into  the  test  and  Integration 
phase.  Lack  of  either  should  be  suffi¬ 
cient  reason  for  delaying  validation 
testing.  Validation  testing  cannot  be 
performed  without  approved  test  pro¬ 
cedures  and  should  not  be  performed 
without  adequate  Internal  configuration 
management  techniques. 

The  CPDP  should  be  reviewed  at  this 
point  to  assure  that  the  contractor 
organization  is  same  as  described  and 


6.1.3  Code  and  Checkout  Phase 


71 


that  the  independence  described  in  the 
CPDP  is  maintained. 

6.1.4  Test  and  Integration  Phase 

Significant  documents  produce'*  during 
this  phase  are  updated  program  ’'stings, 
test  reports  that  show  the  results  of 
all  qualification  tests,  and  a  draft  of 
the  user  manual.  Test  reports  are  sig¬ 
nificant  inputs  to  the  FCA.  These  docu¬ 
ments  are  the  primary  evidence  presented 
at  FCA  for  evaluating  whether  computer 
programs  perform  according  to  the 
development  and  product  specifications. 
A  draft  of  the  user's  manual  should  be 
produced  and  validated  during  this 
phase.  Preparation  of  the  user  manual 
during  this  phase  permits  enough  time  to 
validate  the  procedures  contained 
therein  during  both  this  phase  and  the 
installation  phase. 

Again,  the  CPDP  should  be  reviewed  to 
assure  that  the  configuration  management 
activities  are  in  accordance  with  the 
CPDP.  Configuration  management  is  of 
great  importance  during  this  phase  to 
establish  the  integrity  of  the  computer 
programs  that  are  eventually  delivered. 

6.1.5  Installation  Phase 

The  installation  phase  culminates  in  the 
FCA  and  the  PCA.  At  this  time  the  com¬ 
puter  programs  have  been  installed  at 
the  government  facility  and  all  documen¬ 
tation  Is  complete  and  current.  Review 
of  the  documentation  should  be  for  com¬ 
pleteness  and  suitability  for  use  in  the 
operation  and  support  phase.  It  is  at 
this  time  that  the  documentation  is 
-  delivered  to  the  Air  Force  and  the  pro¬ 
duct  baseline  is  established.  PCA  and 
FCA  may  be  moved  up  to  the  completion  of 
the  test  and  integration  phase  if  there 
are  no  significant  differences  in  the 
installation  in  which  the  computer  pro¬ 
grams  are  to  be  used. 


6.2  DOCUMENT  EVALUATION 

Delivered  documents  will  vary  in  content 
and  quality  regardless  of  what  is 
prescribed  in  a  DID.  These  differences 
depend  on  the  level  of  importance  the 
contractor  gives  to  documentation,  the 
skill  of  the  engineer  preparing  the 
documents,  the  explicitness  and  appli¬ 
cability  of  the  DID  and  the  insistence 
of  the  project  office  in  demanding 
quality  documentation.  The  importance  of 
good  documentation  has  already  been 
discussed;  it  is  essential  for  efficient 
maintenance  in  the  operation  and  support 
phase.  Because  of  this  variance  in 
quality,  each  of  the  documents  on  the 
CDRL  should  be  reviewed  and  should  be 
accepted  only  if  the  document  satisfies 
the  purpose  of  the  intended  users.  It 
must  be  understood  that  good  documenta¬ 
tion  is  expensive  and  is  sometimes  a 
negotiable  item.  Since  the  primary  use 
for  delivered  documentation  is  for 
operation  and  support,  the  appropriate 
commands  should  be  involved  in  any 
decision  to  delete  requirements  for 
particular  documents  from  any  contract. 

An  important  factor  in  both  the  quality 
of  a  document  and  its  evaluation  is  the 
DID  itself.  The  importance  of  tailoring 
the  DID  for  the  specific  application  has 
been  discussed  in  the  previous  section. 
If  a  DID  is  confusing,  contains  irrele¬ 
vant  material  or  is  not  applicable,  the 
resulting  documents  will  surely  suffer. 

As  shown  in  figure  6.1-1,  most  of  the 
documents  that  are  to  be  delivered 
appear  in  draft  form  prior  to  official 
delivery.  This  provides  an  opportunity 
for  reviewing  the  documents  prior  to 
delivery  at  PCA.  The  development 
specifications  for  ATE  software  and  the 
computer  program  development  plan  are 
partial  exceptions.  They  are  written  and 
delivered  in  the  early  stages  of  devel¬ 
opment.  Review  of  these  documents  is 
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automatic  because  ..they  define  contrac¬ 
tual  obligations  for  computer  program 
performance  and  the  plan  for  developing 
the  software.  These  documents  receive 
much  attention  and  their  development  is 
closely  monitored  and  negotiated. 

6.2.1  Evaluation  Criteria 

Each  document  should  be  evaluated 
according  to  the  following  criteria: 

a.  Does  it  satisfy  the  DID? 

b.  Is  it  clearly  and  concisely 
written? 

c.  Is  it  technically  accurate? 

d.  Does  it  completely  satisfy  the 
purpose  of  the  document? 

e.  Are  interfaces  clearly  defined? 

6. 2. 1.1  Does  the  Document  Satisfy  the 
DID?  DIDs  are  descriptions  of  documents 
that  are  required,  by  contract,  to  be 
delivered  to  the  Air  Force.  Most  DIDs 
state  explicitly  the  format  and  content 
of  the  document.  CDRL  documents  should 
be  evaluated  primarily  as  to  whether  the 
document  'being  reviewed  complies  with 
the  DID.  Most  contractor- prepared  docu¬ 
ments  will  comply  with  the  format,  at 
least  in  the  paragraph  or  section 
headings.  The  important  thing  is  to 
determine  whether  the  intention  of  the 
DID  is  satisfied.  Assuming  the  DID  is 
properly  written  the  intention  and  use 
of  the  document  should  be  clear.  If  the 
document  does  not  satisfy  the  intention 
of  the  DID  it  should  be  returned  to  the 
contractor  with  explicit  comments  as  to 
why  it  was  rejected  and  where  it  did  not 
comply  with  the  DID. 

6. 2. 1.2  Is  the  Document  Clearly  and 
Concisely  Written?  Another  major  point 
to  be  evaluated  is  whether  the  document 
can  be  easily  read,  understood,  and  used 
by  the  eventual  user  of  the  document. 
Documents  should  be  written  in  such  a 
manner  that  procedures  and  descriptions 


are  clear  and  unambiguous.  Text  should 
be  written  in  simple  English  with  terms 
familiar  to  the  user.  Logic  flows, 
whether  they  are  flow  charts,  PDL  or 
some  other  approved  form,  should  be  easy 
to  follow  and  should  conform  to  a 
recognized  standard.  (Standards  pub¬ 
lished  in  the  CPDP  and  approved  by  the 
project  office  are  sufficient.)  Logic 
flows  should  be  well  annotated  and 
should  correlate  directly  with  the 
narrative  and  program  listing.  In  turn 
program  listings  should  be  sufficiently 
well  annotated  to  permit  a  direct  corre¬ 
lation  to  narrative  and  logic  flows. 
Requirements  expressed  in  development 
specifications  should  be  clearly  stated 
and  unambiguous.  Eash  requirement  should 
be  singularly  stated  singularly  identi¬ 
fied  and  should  be  testable. 

6. 2. 1.3  Is  the  Document  Technically 

Accurate?  Technical  accuracy  may  be 
difficult  to  evaluate  by  other  than 
engineers  who  have  been  closely 
following  the  computer  program  devel¬ 
opment  process.  Special  emphasis  should 
be  placed  on  the  documents  that 
describe  the  as-built/as-delivered 
computer  programs  and  the  procedures  for 
maintenance  and  support.  The  test 
process  should  verify  that  the  computer 
program  product  specification  accurately 
describes  the  delivered  computer  pro¬ 
gram.  Operating  procedures  should  also 
be  validated  during  the  test  process. 
System  generation  procedures  should  be 
accurately  described  and  proven  during 
the  test  and  integration  phase  when  the 
software  system  generation  is  performed 
between  testing  periods.  Assessing 

technical  accuracy  requires  familiarity 
with  the  computer  programs,  and  knowl¬ 

edge  of  tests  performed  and  validation 
of  the  procedures  described. 

6. 2. 1.4  Is  the  Purpose  of  the  Document 

Satisfied?  The  purpose  of  most  of  the 
computer  program  documents  is  either 

stated  in  the  document  or  is  easily 

understood  by  the  title.  A  document  may 
have  multiple  users,  each  with  a 
different  purpose  in  mind.  The  reviewer 
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should  attempt  to  place  himself  in  the 
user's  position  to  evaluate  whether  all 
the  purposes  of  the  document  are 
satisfied.  For  example,  the  product 
specification  is  used  for  configuration 
management  purposes  as  the  configuration 
Identification  of  the  product  baseline. 
It  is  also  used  by  support  personnel  as 
the  primary  document  describing  the  com¬ 
puter  program  structure  for  maintenance 
purposes.  The  use  and  users  should  have 
been  identified  in  the  CDRL  selection 
process  (paragraph  5.1).  If  the  document 
does  not  satisfy  user  requirements  the 
DID  should  be  examined  to  determine  if 
it  needs  to  be  modified.  If  the  DID 
needs  to  be  modified  a  contractual 
change  may  be  required  to  respond  to  the 
DID  modification.  The  change  should  be 
noted  and  included  in  DIDs  for  similar 
future  acquisition. 

6. 2. 1.5  Are  Interfaces  Clearly 
Defined?  Poor  definition  of  interfaces 
is  poor  design  and  leads  to  confusion 
during  the  design,  coding,  test  and 
integration  and  support  phases.  Inter¬ 
faces  between  computer  program  (CPCIs) 
and  hardware  external  to  the  program  are 
vital  to  good  communication  between 
different  groups.  This  includes  data 
formats,  data  rates,  methods  of 
initiating  data  transfer  and  use  of  I/O 
channels.  The  data  base  is  the  primary 
means  of  transferring  data  between 
computer  program  components.  The  tables, 
files  and  ..other  structures  should  be 
identified  and  described  in  detail. 
Access  methods,  names,  and  naming  con¬ 
ventions  should  be  defined. 

Document  evaluation  should  never  be 
taken  lightly.  Before  a  document  is 
accepted  by  the  Air  Force  it  should  have 
been  comprehensively  reviewed  and  all 
deficiencies  corrected.  Table  6.2-1  is  a 
summary  check  for  document  evaluation. 

6.3  DOCUMENT  REVISION 

Once  a  document  is  completed  and 
released.  It  should  be  subjected  to  some 
measure  of  formal  change  control  to 


prevent  Indiscriminate  revision.  This 
control  may  be  entirely  internal  to  the 
contractor  or  may  require  project  office 
action.  Figures  6.3-1  and  6.3-2  show  the 
evolution  of  the  development  of  TS  sys¬ 
tems  and  ATE  computer  program  documents 
and  the  change  control  categories  to 
which  they  are  assigned.  In  general, 
documents  are  maintained  under  con¬ 
tractor  control  methods  after  they  have 
been  released  and  remain  so  until  formal 
delivery  to  the  Air  Force.  After  formal 
delivery,  changes  to  documents  Identi¬ 
fying  computer  program  configuration 
baselines  and  maintenance  operating 
instructions  require  approval  by  the  Air 
Force.  The  PDR,  CDR,  FCA  and  PCA  shown 
on  the  figure  are  representative  of  a 
single  CPCI  or  Cl.  Each  CPCI  or  Cl  is 
scheduled  separately  with  its  own  mile¬ 
stones  because  the  need,  availability 
and  resource  may  differ  for  each  CPCI. 

6.3.1  Document  Change  Control 

Two  levels  of  change  control  are 
applicable  to  the  documents  produced  for 
computer  program  development.  They  are 
formally  des ingated  as  Class  I  and  Class 
II.  Complete  definitions  of  these  two 
change  classifications  are  found  in 
MIL-STD-483.  In  general.  Class  I  changes 
require  Air  Force  approval  and  Class  II 
changes  require  only  Air  Force  concur¬ 
rence  that  the  changes  are  not  Class  I. 
Thus,  Class  II  changes  are  controlled  by 
the  contractor.  Figures  6.3-1  and  6.3-2 
show  the  transition  points  from  con¬ 
tractor  control  to  Air  Force  control  for 
the  primary  documents  described  in  this 
guidebook.  In  the  earliest  stages  of 
development,  changes  to  the  Computer 
Program  Development  Specification,  the 
CPDP,  the  CMP  and,  for  TS  systems,  the 
contractor  proposal  require  Air  Force 
approval.  As  the  remaining  documents  are 
written  and  released  they  should  be 
placed  under  some  formal  means  of  con¬ 
trol  by  the  contractor.  The  remaining 
documents  are  delivered  to  the  Air  Force 
at  PCA.  Those  to  be  used  later  in  the 
Operations  and  Support  phase  are  then 
placed  under  Class  I  change  control. 
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Tabia  6.2-1.  Documant  Evaluation  Chacklitt 


1.  Does  the  document  satisfy  the  DID? 

2.  Is  the  document  written  clearly  and  concisely? 

3.  Are  procedures  described  in  a  logical  step-by-step  process? 

4.  Are  all  statements  clear  and  unambiguous? 

5.  Do  logic  flows  clearly  describe  the  process  intended? 

6.  Is  there  a  clear  correlation  between  logic  flows,  narrative  text  and 
program  listings? 

7.  Are  standards  for  logic  flows  defined? 

8.  Are  requirements  expressed  clearly  and  singularly? 

9.  Are  requirements  testable? 

10.  Is  the  document  technically  accurate? 

11.  Do  test  results  confirm  the  design  description? 

12.  Have  operating  procedures  been  validated? 

13.  Have  system  generation  procedures  been  validated? 

14.  Are  all  purposes  of  the  document  satisfied? 

15.  Have  all  document  users  been  identified,  have  their  needs  been  considered? 

16.  Are  all  external  interfaces  explicitly  defined? 

17.  Have  all  data  base  elements  been  defined  and  adequately  described? 

18.  Is  it  clear  how  transfer  of  data,  whether  external  or  internal  to  the 
program,  are  controlled? 


6.3.2  Change  Control  During  Develop¬ 
ment 

During  the  development  process  only 
those  documents  relating  to  defining  the 
allocated  baseline  and  planning  for  com¬ 
puter  program  development  and  configu¬ 
ration  management  are  under  Class  I 
control.  ATE  development  specifications 
ar’e  normally  generated  by  the  contractor 
and,  upon  approval  by  the  Air  Force,  are 
placed  under  Class  I  change  control.  The 
system  specification  and  the  contractor 
proposal  provide  the  equivalent  baseline 
for  TS  systems.  The  CMP  and  the  CPDP 
which  may  be  generated  as  part  of  the 
contractor's  proposal  (TS)  or  early  in 
the  development  phase  (ATE)  and  upon 
approval  become  part  of  the  contract. 
Changes  to  these  documents  frequently 
affect  cost,  performance  or  both. 
Changes  are  initiated  by  an  ECP,  usually 
by  the  contractor.  They  are  reviewed  and 
approved  (or  disapproved)  by  the  contrac¬ 
tor  change  board  and  submitted  to  the 
Air  Force  change  board  for  review  and 
approval  (or  disapproval).  Upon  Air 
Force  approval  the  affected  documents 
are  updated  by  SCN  or  Document  Change 
Notice  (DCN)  and  appropriate  action 
taken. 

As  the  remaining  documents  are  developed 
and  released,  change  control  is  in  the 
contractor's  hands.  It  is  important  that 
the  contractor's  document  change,  control 
procedures  are  known  and  observed.  These 
procedures  should  be  provided  in  the 
CPDP.  The  contractor  may  have  several 
levels  of  Class  II  classification  per¬ 
mitting  efficient  processing  for  docu¬ 
ment  and  design  changes.  The  lowest 
level  changes  may  require  only  approval 
by  the  software  supervisor,  the  higher 
levels  by  the  program  change  board.  It 
is  particularly  important  for  computer 
program  design  documentation  to  be  kept 
current  with  the  computer  program  code 


during  the  test  and  integration  phase  to 
enable  accurate  tracing  of  the  configura¬ 
tion  of  the  programs  finally  validated 
and  to  assure  that  the  code  matches  the 
design  documentation.  A  contractor  ver¬ 
sion  of  a  VDD  could  be  used  effectively 
to  trace  these  changes. 

6.3.3  Changes  During  the  Operation  and 
Support  Phase 

Changes  during  the  operation  and  support 
phase  are  complicated  by  the  fact  that 
TS  systems  and  ATE  may  be  installed  at 
several  sites.  If  strong  configuration 
management  techniques  are  not  imposed 
this  may  result  in  different  versions  of 
the  computer  programs  at  the  operational 
sites.  Changes  to  computer  programs 
(code)  and  their  documentation  should  be 
controlled  from  one  central  point. 
Operation  and  support  of  the  computer 
programs  during  this  phase  is  normally 
provided  by  Air  Force  personnel  but  may 
also  be  provided  by  a  contractor. 


Change  procedures  are  similar  to  Class  I 
procedures  used  during  development. 
Changes  are  normally  requested  by  the 
using  command  when  discrepancies  are 
discovered,  when  design  changes  are 
needed  to  improve  operations  or  perfor¬ 
mance,  or  when  there  are  changes  to  the 
weapon  systems  being  simulated  (TS)  or 
changes  to  the  units  being  tested  (ATE). 
Approved  requests  are  passed  on  to  the 
support  command  or  contractor  for  incor¬ 
poration  into  a  new  version  of  the  com¬ 
puter  program.  Documentation  changes 
should  always  accompany  changes  to  the 
computer  program.  The  new  version  should 
not  be  released  until  documentation  is 
complete  and  available  to  all  users.  The 
support  command  is  the  responsible 
organization  for  maintaining  both  the 
computer  program  and  the  program  docu¬ 
mentation. 
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Section  7.0  DOCUMENT  USAGE 


Document  usage,  one  of  the  factors  In 
the  selection  of  an  appropriate  CDRL, 
was  discussed  In  Section  5.0.  It  Is 
Important  that  each  document  requested 
by  the  CDRL  have  a  specific  purpose  and 
that  It  Is  needed  by  the  Air  Force.  That 
Is  to  say,  the  CDRL  should  be  the  result 
of  a  great  deal  of  thought  and  design. 
The  primary  consideration  Is  the  use 
that  will  be  made  of  the  documents  by 
the  Air  Force.  Computer  program  documen¬ 
tation,  whether  it  is  prepared  by  the 
Air  Force  or  by  the  development  contrac¬ 
tor,  is  prepared  for  one  of  the  uses 
listed  below: 

a.  System  requirements  and  acquisi¬ 
tion  management. 

b.  Procurement 

c.  Configuration  management 

d.  Engineering 

e.  Test 

f.  Operation  and  support 

Documentation  for  the  first  two  uses  is 
usually  prepared  by  the  Air  Force  and 
the  last  four  by  the  development  con¬ 
tractor.  Documents  used  for  these  pur¬ 
poses  are  discussed  In  the  following  sec¬ 
tions.  Documents  that  have  multiple 
purposes  will  be  discussed  in  more  than 
one  category.  Since  specific  documents 
are  prepared  for  the  same  purposes, 
whether  for  TS  or  ATE  computer  programs, 
they  are  discussed  together.  Different 
emphases  are  highlighted  where  neces¬ 
sary. 

7.1  SYSTEM  REQUIREMENTS  AND  ACQUISITION 
MANAGEMENT  DOCUMENTS 

Documents  prepared  for  this  use  are  the 
ROC,  PMD,  PMP,  CRISP,  and  the  CPDP. 
These  documents  are  all  prepared  by  the 
Air  Force  with  the  probable  exception  of 
the  CPDP.  The  CPDP  may  be  prepared  by 


the  Air  Force  or  more  commonly  by  a  con¬ 
tractor.  These  documents  are  used  by  the 
program  manager  and  other  participating 
Air  Force  organizations  to  define  pro¬ 
gram  concepts,  agreements  and  planned 
action. 

The  ROC  and  the  PMD  define  overall  sys¬ 
tem  requirements  and  provide  for  (1) 
agreements  between  the  using,  develop¬ 
ment,  and  support  commands  regarding 
those  requirements;  (2)  management 
responsibility  for  the  acquisition  of 
subsystems  containing  computer  equipment 
and  computer  programs;  and  (3)  for  the 
integration  of  computer  programs  and  com¬ 
puter  equipment  into  overall  systems. 
The  ROC  and  PMD  are  very  likely  prepared 
specifically  for  TS  acquisition;  thus, 
management  attention  is  focused  on  the 
TS  and  the  computing  hardware  and  com¬ 
puter  programs.  Normally,  ATE  is 
included  in  the  ROC  and  PMD  for  an 
entire  weapon  system  and  since  the  time 
between  the  ROC  and  ATE  acquisition  is 
great,  few  specifics  are  included. 

The  PMP,  CRISP  and  CPDP  are  the  primary 
documents  for  acquisition  management. 
The  PMP  is  prepared  specifically  for  TS 
system  acquisitions  and  for  the  weapon 
system  for  ATE  acquisitions.  The  CRISP 
and  the  CPDP  are  written  specifically 
for  the  acquisition  of  TS  and  ATE. com¬ 
puter  programs.  When  ATE  is  added  to  the 
weapon  system  contract,  the  PMP  must  be 
updated.  The  PMP  provides  planning  and 
resource  management  for  the  entire  TS  or 
ATE  system  dffHng  the  development  phase 
while  the  CRISP  provides  for  computer 
resource  management  in  the  development 
phase  and  through  the  transition  to  the 
operation  and  support  phase.  The  CRISP 
is  a  living  document  and  is  maintained 
and  updated  throughout  the  computer 
program  life  cycle.  These  documents,  PMP 
and  CRISP,  provide  a  plan  for  the  pro¬ 
gram  manager  that  has  been  coordinated 
with  the  developing,  using  and  support 
commands. 
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The  CPDP  is  usually  written  by  the  devel¬ 
opment  contractor  and  describes  his  plan 
for  developing  ATE  or  TS  computer  pro¬ 
grams.  The  TS  CPDP  is  prepared  as  part 
of  the  contractor's  proposal  and  can  be 
used  in  the  source  selection  process. 
The  ATE  CPDP  is  prepared  after  negotia¬ 
tions  are  complete  on  the  contract  exten¬ 
sion.  In  either  case,  the  CPDP  when 
negotiated  and  approved  by  the  Air  Force 
can  become  a  contractual  document  which 
must  be  observed  by  the  contractor. 

7.2  PROCUREMENT  PLANNING  DOCUMENTS 

Documents  prepared  for  procurement  pur¬ 
poses  are  written  by  the  Air  Force  and 
include  the  RFP  and  the  SFE.  The  RFP 
includes  the  SOW,  CDRL,  IFPP  and  TS 
system  specifications.  Procurement  plan¬ 
ning  documents  are  prepared  for  the  TS 
systems  acquisition  and  are  only 
remotely  applicable  for  ATE  systems 
unless  it  is  acquired  under  a  separate 
contract.  The  same  documents  are  also 
required  for  a  weapon  system  procurement 
but  probably  will  not  specifically 
involve  ATE. 

The  RFP  is  a  collection  of  documents  pre¬ 
pared  by  the  Air  Force  for  the  purpose 
of  describing  the  job  to  be  bid.  The  SOW 
describes  the  tasks  to  be  performed  by 
the  contractor.  The  CDRL  is  a  list  of 
documents  and  other  data  to  be  provided 
by  the  contractor  to  the  Air  Force  in 
fulfillment  of  the  contract.  Both  the 
SOW  and  the  CDRL  may  be  modified  by  the 
contractor  in  his  proposal.  After  con¬ 
tract  award,  the  SOW  and  CDRL  will  be 
negotiated  and  will  result  in  a  con¬ 
tractual  agreement  regarding  the  tasks 
to  be  performed  and  the  data  to  be 
delivered.  The  TS  system  specification 
will  be  used  to  define  the  functional 
and  performance  requirements  for  the  TS 
system.  These  are  requirements  that  the 
bidders  are  required  to  satisfy  in  their 
proposal.  The  IFPP  provides  detailed 
instructions  as  to  how  the  proposal  is 
to  be  prepared,  what  it  is  to  cover  and 
what  the  contents  of  the  various  parts 
of  the  proposal  should  be.  The  IFPP  is 


to  be  used  by  the  bidders  for  consistent 
preparation  of  proposals  that  can  be 
evaluated  fairly.  !.  should  address  all 
the  elements  by  which  the  proposal  is  to 
be  evaluated.  The  evaluation  factors  are 
written  in  the  SFE.  The  SFE  is  used  by 
the  Air  Force  review  team  to  define  how 
the  proposals  will  be  evaluated  to 
ensure  a  consistent  evaluation  and  award 
recommendation.  The  SFE  is  not  available 
to  the  contractors  but  should  be 
reflected  in  the  IFPP. 

7.3  CONFIGURATION  MANAGEMENT  DOCUMENTS 

Configuration  management  is  concerned 
with  the  identification  of  configuration 
baselines,  configuration  change  control 
and  configuration  accounting.  Documenta¬ 
tion  used  for  configuration  management 
must  provide  for  these  three  attributes. 
Documents  used  for  configuration  manage¬ 
ment  are  as  follows:  CMP,  computer  pro¬ 
gram  development  specification,  CPDP, 
computer  program  product  specification, 
interface  design  description,  SCN, 
change  status  list,  configuration  index, 
and  VDD. 

The  CMP  is  prepared  during  the  proposal 
period  and  usually  becomes  a  contractual 
obligation  on  the  contractor.  It  is  used 
by  the  Air  Force  project  manager  to 
ensure  configuration  management  is  per¬ 
formed  according  to  the  contractor's 
plan.  It  is  used  by  the  contractor  as 
the  master  plan  for  configuration  manage¬ 
ment  activities.  While  it  addresses  com¬ 
puter  programs,  at  least  for  the  periods 
prior  to  delivery  to  the  Air  Force,  are 
found  in  the  CPDP.  The  CMP  is  developed 
as  part  of  the  weapon  system  proposal 
for  ATE  and  as  part  of  the  TS  system  pro¬ 
posal  . 

The  documents  identifying  the  computer 
program  configuration  baselines  are  the 
computer  program  development  specifica¬ 
tion,  the  computer  program  product  speci¬ 
fication,  and  the  interface  design 
description  document.  Once  established, 
a  configuration  management  baseline  Is 
placed  under  Class  I  change  control  and 


requires  Air  Force  approval  for  any 
changes.  The  development  specification 
when  authenticated  becomes  the  CPCI  allo¬ 
cated  baseline  which  Is  the  "design  to" 
requirements  baseline.  This  baseline 
establishes  the  functional  and  perfor¬ 
mance  requirements  which  must  be  satis¬ 
fied  before  ATE  computer  programs  or  the 
TS  system  is  accepted  by  the  Air  Force. 
The  TS  system  development  specification 
is  prepared  by  the  Air  Force  and  imposed 
on  the  contractor  at  contract  award. 
Development  specifications  are  prepared 
by  the  contractor  for  each  ATE  CPCI.  The 
product  specification  and  the  interface 
design  description  document  identify  the 
product  baseline  which  is  the  "as-built" 
configuration.  These  documents  describe 
in  detail  by  means  of  logic  flow 
diagrams,  written  text,  and  program  list¬ 
ings,  the  exact  approved  configuration 
of  the  computer  programs  delivered  to 
the  Air  Force.  The  one  to  one  correla¬ 
tion  of  the  product  specification  with 
respect  to  the  computer  program  code 
should  be  established  and  verified  at 
the  PCA  prior  to  delivery  to  the  Air 
Force. 

The  Configuration  Index,  Change  Status 
List,  SCN,  and  the  VDD  provide  the  means 
of  identifying  changes  to  the  ATE  and  TS 
computer  program  configurations;  the 
identification  and  status  of  impending 
changes  and  the  complete  identification, 
by  program  component,  of  all  approved 
configurations.  Change  control  proce¬ 
dures  are  described  in  the  CPDP  and  the 
CMP.  Actual  changes  to  the  computer  pro¬ 
gram  components  are  recorded  in  the  pro¬ 
duct  specification. 

7.4  ENGINEERING  DOCUMENTS 

Engineering  documentation  is  used  for 
progressive  definition  of  technical  per¬ 
formance  and  for  relating  performance 
requirements  to  the  design  definition. 
CDRL  documents  fulfilling  these  needs 
are:  preliminary  and  draft  versions  of 
the  computer  program  product  specifica¬ 


tion  and  the  interface  design  descrip¬ 
tion  documents,  the  TRD  and  the  computer 
programming  manual. 

Preliminary  and  draft  product  specifi¬ 
cation  and  the  interface  description 
documents,  produced  for  PDR  and  CDR 
respectively,  record  the  progressive 
definition  of  the  TS  and  ATE  computer 
program  design.  Without  a  written 
design,  there  truly  is  no  design  at  all. 
The  design  progressively  recorded,  pro¬ 
vides  opportunity  for  review,  comments, 
and  for  coordination  between  designers. 
Maintaining  these  engineering  data  in  an 
up-to-date  status  is  vital  for 
continuing  review,  comparison  with 
requirements  and  accurate  communication. 

The  TRD  is  the  result  of  design  engineer¬ 
ing  and  factory  qualification  testing 
for  a  unit  to  be  tested.  It  specifies 
the  requirements  for  testing  that  unit. 
The  TRD  does  not  qualify  as  a  configura¬ 
tion  management  baseline,  but  the  data 
contained  therein  is  the  basis  for  ATE 
test  software,  thus  it  is  classified  as 
an  engineering  document.  The  early  TRD 
data  is  also  used  an  input  to  the  sup¬ 
port  equipment  analysis  that  results  in 
the  support  equipment  requirements  data 
analysis. 

The  computer  programming  manual  provides 
technical  data  necessary  to  enable  an 
experienced  programmer  to  write  computer 
programs  for  a  given  computer  in  a 
designated  programming  language.  The  com¬ 
puter  programming  manual  is  specified  in 
ATE  CDRL  and  is  a  part  of  the  TS.  The 
computer  programming  manual  may  not  be 
prepared  until  the  coding  and  checkout 
phase  is  completed  but  the  information 
is  necessary  for  efficient  programming 
practices. 

7.5  TEST  DOCUMENTS 

Test  documentation  is  used  for  the  quali¬ 
fication  and  acceptance  of  equipment  and 
computer  programs,  and  to  provide  the 
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basis  for  testing  future  modifications. 
Documents  satisfying  these  uses  are  test 
plans,  test  procedures  and  test  reports. 
Even  though  qualification  and  acceptance 
of  TS  computer  programs  Is  accomplished 
at  the  TS  system  level,  test  documenta¬ 
tion  Is  necessary  for  computer  programs. 
Test  plans,  whether  they  are  for  the  TS 
system  or  for  ATE  computer  programs,  pro¬ 
vide  for  the  planning,  scheduling  and 
coordination  of  computer  programs,  com¬ 
puters,  and  interfacing  hardware  neces¬ 
sary  for  conducting  qualification  test¬ 
ing.  Test  procedures  provide  the  step- 
by-step  sequence  of  events  for  each  test¬ 
ing  for  adequacy  and  for  Air  Force 
concurrence  with  the  test  procedures. 
They  also  provide  a  record  of  the  tests 
and  procedures  actually  performed.  Test 
reports  provide  the  evidence  that  tests 
were  performed  and  make  a  permanent 
record  for  pertinent  test  results. 

7.6  OPERATION  AND  SUPPORT  DOCUMENTS 

Operation  and  support  documentation  is 
used  to  operate,  maintain,  modify  and 
otherwise  support  the  system  after 
acceptance.  Documents  used  for  opera¬ 
tions  and  support  are  as  follows:  users 
manual  (ATE),  maintenance  manual  (ATE), 
computer  programming  manual  (ATE),  test 
equipment  computer  program  documentation 
(TS),  computer  program  product  specifica¬ 


tions,  interface  design  documents  and 
the  VDD.  The  users  manual  and  computer 
programming  manual  for  ATE  and  the  test 
equipment  computer  program  documentation 
for  TS  systems  provide  the  technical 
information  needed  for  the  using  command 
to  load  the  computers  with  the  appro¬ 
priate  computer  programs,  bring  it  into 
an  operating  status,  control  the  opera¬ 
tion  of  the  computer  programs,  intitlate 
recovery  procedures  and  provide  trouble¬ 
shooting  techniques  for  diagnosing  prob¬ 
lems.  It  also  provides  data  for  the  T.O. 
used  to  conduct  testing. 

The  support  command  uses  the  users 
manual  and  the  computer  programming 
manual  for  ATE  or  the  training  equipment 
computer  program  documentation  for  tech¬ 
nical  information  regarding  the  computer 
and  the  programming  languages  used,  main¬ 
tenance  procedure  and  computer  program 
system  generation.  The  computer  program 
product  specification  and  the  interface 
design  documents  provide  the  current 
design  data  and  descriptions  necessary 
to  accomplish  changes  to  the  computer 
programs.  The  VDD  provides  the  exact 
configuration  of  all  operating  or  experi¬ 
mental  versions  of  the  TS  or  ATE  com¬ 
puter  programs.  Approved  configurations 
can  be  used  with  knowledge  of  the  exact 
changes  that  are  included. 
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Section  9.0  GUIDEBOOK  TOPICS  VS  GOVERNMENT  DOCUMENTS  CROSS' REFERENCE 


Figure  9.0-1  is  a  cross  reference  matrix 
showing  the  guidebook  topics  and  govern¬ 
ment  documents  that  address  that  topic. 
The  government  documents  are  identified 
as  well  as'  the  sections,  chapters, 
attachments,  enclosures,  appendicies. 


etc.  in  which  the  topics  are  found.  When¬ 
ever  a  guidebook  topic  Is  also  the  pri¬ 
mary  subject  of  a  government  document,  a 
bullet  is  used  to  designate  the  intersec¬ 
tion. 
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Section  10.0  GLOSSARY  OF  TERMS 


A1 1 ocated  Basel 1 ne  -  The  approved  con¬ 
figuration  item  identification.  It 
governs  the  development  of  selected  con¬ 
figuration  items  that  are  part  of  a 
higher  level  specification,  e.g. ,  system 
specification.  It  is  usually  defined  by 
the  Computer  Program  Development  Specifi¬ 
cation. 

Basel i ne  -  An  authorized  documented  tech- 
nical  description  specifying  an  end 
item’s  functional  and  physical  character¬ 
istics.  It  serves  as  the  basis  for  con¬ 
figuration  control  and  status  account¬ 
ing.  It  establishes  an  approved  well- 
defined  point  of  departure  for  control 
of  future  changes  to  system  or  equip¬ 
ment. 

Certification  -  The  test  and  evaluation 
of  the  complete  computer  program  aimed 
at  ensuring  operational  effectiveness 
and  suitability  with  respect  to  mission 
requirements  under  operating  conditions. 

Computer  Program  -  A  series  of  Instruc¬ 
tions  or  statements  in  a  form  acceptable 
to  computer  equipment,  designed  to  cause 
the  execution  of  an  operation  or  series 
of  operations.  Computer  programs  Include 
such  Items  as  operating  systems,  utility 
programs,  and  maintenance/diagnostic  pro¬ 
grams.  They  also  include  applications 
programs  such  as  payroll,  inventory,  con¬ 
trol,  operational  flight,  strategic,  tac¬ 
tical,  automatic  test,  crew  simulator 
and  engineering  analysis  programs.  Com¬ 
puter  programs  may  be  either  machine 
dependent  or  machine  Independent,  and 
may  be  general  purpose  in  nature  or  be 
designed  to  satisfy  the  requirements  of 
a  specialized  process  of  a  particular 
user. 

Computer  Program  Development  Cycle  -  The 
computer  program  development  cycle  con¬ 
sists  of  six  phases:  analysis,  design, 
coding  and  checkout,  test  and  Integra¬ 
tion,  installation,  and  operation  and 
support.  The  cycle  may  span  more  than 
one  system  acquisition  life  cycle  phase 
or  may  be  contained  In  any  one  phase. 
(AFR  800-14,  Volume  II). 


Computer  Program  Configuration  Items  -  A 
computer  program  or  aggregate  of  related 
computer  programs  designated  for  config¬ 
uration  management.  A  CPCI  may  be  a 
punched  deck  of  cards,  paper  or  magnetic 
tape  or  other  media  containing  a 
sequence  of  instructions  and  data  in  a 
form  suitable  for  insertion  in  a  digital 
computer. 

Configuration  Item  -  An  aggregation 
which  satisfies  an  end  use  function  and 
is  designated  for  configuration  manage¬ 
ment. 

Configuration  Management  -  A  management 
discipline  applying  technical  and  admin¬ 
istrative  direction  and  surveillance  to: 

a.  Identify  and  document  the  func¬ 
tional  and  physical  characteristics  of  a 
configuration  item; 

b.  Control  changes  to  those  charac¬ 
teristics;  and 

c.  Record  and  report  change  process¬ 
ing  and  implementation  status. 

Control  Software  -  Software  used  during 
execution  of  a  test  program  which  con¬ 
trols  the  nontesting  operations  of  the 
ATE.  This  software  Is  used  to  execute  a 
test  procedure  but  does  not  contain  any 
of  the  stimuli  or  measurement  parameters 
used  In  testing  a  unit  under  test.  Where 
test  software  and  control  software  are 
combined  In  one  inseparable  program, 
that  program  will  be  treated  as  test 
software.  (AFLC  66-37) 

Data  Base  -  A  collection  of  program 
code,  tables,  constants.  Interface  ele¬ 
ments  and  other  data  essential  to  the 
operation  of  a  computer  program  or  soft¬ 
ware  subsystem. 

External  Interface  -  Data  passed  between 
two  or  more  computer  programs  or  between 
a  computer  program  and  peripheral 
devices  external  to  the  computer  In 
which  the  program  resides.  The  data  m«y 
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be  in  the  form  of  an  interrupt  signal  or 
may  be  a  digital  data  stream  either  out¬ 
put  from  the  computer  or  input  into  the 
computer  for  processing. 

Internal  Interfaces  -  Data  passed 
between  elements  of-  a  computer  program 
and  usually  included  in  the  computer  pro¬ 
gram  data  base. 

Logic  Flow  -  A  diagrammatic  representa¬ 
tion  of  the  logic  sequence  for  a  com¬ 
puter  program.  Logic  flows  may  take  the 
form  of  the  traditional  flow  charts  or 
i n  some  other  form  such  as  a  program 
design  language. 

Organic  -  A  term  used  to  designate  a 
task  performed  by  the  Air  Force  rather 
than  a  contractor. 

Product  Baseline  -  The  final  approved 
configuration  identification.  It  identi¬ 
fies  the  as  designed  and  functionally 
tested  computer  program  configuration. 
It  is  defined  by  the  Computer  Program 
Product  Specification. 

Program  Design  Language  -  An  English- 
like,  specifcally  formatted,  textual 
language  describing  the  control  struc¬ 
ture,  logic  structure,  and  general 
organization  of  a  computer  program. 
Essential  features  of  a  program  design 
language  are: 

a.  It  is  an  English-like  representa¬ 
tion  of  a  computer  procedure  that  is 
easy  to  read  and  comprehend. 

b. '  It  is  structured  in  the  sense  that 
it  utilizes  the  structured  programming 
control  structures  and  indentation  to 
show  nested  logic. 

c.  It  uses  full  words  or  phases 
rather  than  the  graphic  symbols  used  in 

flowcharts  and  decision  tables. 
i 

Qual'ity  Assurance  -  A  planned  and  sys¬ 
tematic  pattern  of  all  software  related 
actions  necessary  to  provide  adequate 
confidence  that  computer  program  con¬ 
figuration  items  or  products  conform  to 


establish  software  technical  require¬ 
ments  and  that  they  achieve  satisfactory 
performance. 

Software  -  A  combination  of  associated 
computer  programs  and  computer  data 
required  to  enable  the  computer  equip¬ 
ment  to  perform  computational  or  control 
functions. 

Software  Quality  -  The  primary  charac¬ 
teristic  of  software  quality  is  that  the 
software  reflects  the  specification  to 
which  it  is  written  but  also  that  the 
software  specifications  themselves  ade¬ 
quately  address  the  system/mission 
requirements.  Key  attributes  of  software 
quality  include:  reliability,  flexi¬ 
bility,  traceability,  testability,  inte¬ 
grity,  maintainability,  and  complete¬ 
ness.  Quality  software  is:  well-defined, 
wel 1 -documented,  free  of  design  defi¬ 
ciencies  and  coding  errors,  satisfies 
performance  requirements,  and  has  mini¬ 
mum  life  cycle  cost. 

Support  Software  -  Auxiliary  software 
used  to  aid  in  preparing,  analyzing  and 
maintaining  other  software.  Support  soft¬ 
ware  is  never  used  during  the  execution 
of  a  test  program  on  a  tester,  although 
it  may  be  resident  either  on-line  or 
off-line.  Included  are  assemblies,  com¬ 
pilers,  translators,  loaders,  design 
aids,  test  aids,  etc.  (AFLC  66-37 ) 

System  Engineering  -  The  application  of 
scientific  and  engineering  efforts  to 
transform  an  operational  need  or  state¬ 
ment  of  deficiency  into  a  description  of 
systems  requirements  and  a  preferred  sys¬ 
tem  configuration  that  has  been  opti¬ 
mized  from  a  life  cycle  viewpoint.  The 
process  has  three  principle  elements: 
functional  analysis,  synthesis,  and 
trade  studies  or  cost-effectiveness  opti¬ 
mization. 

System  Generation  -  The  process  of  pro¬ 
ducing  a  computer  program  from  two  or 
more  program  elements  such  that  the 
separate  program  elements  will  perform 
together  as  an  integrated  whole. 
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Test  Software  -  Programs  which  Implement 
documented  test  requirements.  There  Is  a 
separate  test  program  written  for  each 
distinct  configuration  of  unit  under 
test.  (AFLC  66-37) 

Top  Down  Integration  -  The  development, 
testing  and  integration  of  the  separate 
structural  elements  of  a  computer  pro¬ 
gram  in  a  hierarchical  manner  beginning 
with  the  highest  levels  and  working  down 
through  the  lowest  levels  until  the  com¬ 
puter  program  is  complete. 

Top  Down  Structured  Programs  -  Struc¬ 
tured  programs  with  the  additional 
characteristics  of  the  source  code  being 
logically,  but  not  necessarily  physi¬ 
cally,  segmented  in  a  hierarchical  man¬ 
ner  and  only  dependent  on  code  already 
written.  Control  of  execution  between 
segments  is  restricted  to  transfer 
between  vertically  adjacent  hierarchical 
segments. 


Val idation  -  Computer  program  validation 
Is  the  test  and  evaluation  of  the  com¬ 
plete  computer  program  aimed  at  ensuring 
compliance  with  the  performance  and 
design  criteria. 

Verification  -  Computer  program  verifi¬ 
cation  is  the  iterative  process  of  con¬ 
tinuously  determining  whether  the  pro¬ 
duct  of  each  step  of  the  computer  pro¬ 
gram  acquisition  process  fulfills  all 
requirements  levied  by  the  previous 
step,  including  those  set  for  quality. 

System  Life  Cycle  -  The  system  acquisi¬ 
tion  life  cycle  consists  of  the  follow¬ 
ing  five  major  phases  with  major  deci¬ 
sion  points: 

a.  Conceptual  phase 

b.  Validation  phase 

c.  Full-scale  development  phase 

d.  Production  phase 

e.  Deployment  phase 

(AFR-800-14,  Volume  II) 
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Section  11.0  ABBREVIATIONS  AND  ACRONYMS 


ADL 

Approved  Data  List 

DOC 

Document 

AFLC 

Air  Force  Logistics  Command 

DOD 

Department  of  Defense 

AFSC 

Air  Force  Systems  Command 

DSARC 

Defense  System  Acquisition 
Review  Council 

AGE 

Aerospace  Ground  Equipment 

ECP 

Engineering  Change  Proposal 

ASD 

Aeronautical  Systems  Division 

ESD 

Electronic  Systems  Division 

ATE 

Automatic  Test  Equipment 

FCA 

Functional  Configuration  Audit 

ATLAS 

Automatic  Test  Language  for 

All  Systems 

FORTRAN 

Formula  Translation 

BCD 

Binary  Coded  Decimal 

HIPO 

Hierarchical  Input  Processing 
Output 

CCP 

Contract  Change  Proposal 

HOL 

Higher  Order  Language 

CDR 

Critical  Design  Review 

HQ 

Headquarters 

CDRL 

Contract  Data  Requirements 

List 

IFPP 

Information  for  Proposal 
Preparation 

Cl 

Configuration  Items 

ILS 

Integrated  Logistics  Support 

CMP 

Configuration  Management  Plan 

ILSO 

Integrated  Logistics  Support 

CP 

Computer  Program 

Office 

CPCI 

Computer  Program  Configuration 

ILSP 

Integrated  Logistics  Support 

Items 

Plan 

CPDP 

Computer  Program  Development 

I/O 

Input /Output 

Plan 

ISP 

Integrated  Support  Plan 

CRISP 

Computer  Resources  Integrated 
Support  Plan 

ITA 

Interface  Test  Adapter 

CRT 

Cathode  Ray  Tube 

LRU 

Line  Replaceable  Unit 

CRWG 

Computer  Resources  Working 

Group 

ORLA 

Optimum  Repair  Level  Analysis 

PCA 

Physical  Configuration  Audit 

DCN 

Document  Change  Notice 

PDL 

Program  Design  Language 

DCP 

Development  Concept  Paper 

PDR 

Preliminary  Design  Review 

DEV 

Development 

PMD 

Program  Management  Directive 

DID 

Data  Item  Description 
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PMP 

Program  Management  Plan 

SW 

Software 

PRELIM 

Preliminary 

SYS 

System 

RFP 

Request  for  Proposal 

TDSP 

Top  Down  Structured 
Programming 

ROC 

Required  Operational 

Capability 

T.E. 

Training  Equipment 

SAE 

Software  Acquisition 

TECPD 

Training  Equipment  Computer 

Engl neerl ng 

Program  Documentation 

SAMSO 

Space  and  Missile  Systems 
Organization 

T.O. 

Technical  Order 

TRD 

Test  Requirements  Document 

SCN 

Specification  Change  Notice 

TS 

Trainer  Simulator 

SEP 

Support  Equipment  Plan 

OUT 

Unit  Under  Test 

SERD 

Support  Equipment 
Recommendation  Data 

USAF 

United  States  Air  Force 

SFE 

Standard  for  Evaluation 

VDD 

Version  Description  Document 

SOU 

Statement  of  Work 

UBS 

Work  Breakdown  Structure 

SPEC 

Specification 

US 

Weapon  System 

System  Program  Office 


SPO 
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5.1.2,  5.4,  5.5,  5.5.1,  5. 5. 1.2,  5. 5.2.1, 

5. 5. 2. 2,  5.5.2. 3,  5.5.2.8,  5.5.2.15, 
5.5.2.16,  5.5.2.18,  5.5.2.19,  5.5.2.20.1, 

5.5.2.20.4,  6.2,  6.2. 1.1,  7.2 

5.5.1.15,  5.5.2.21 
5.0,  5.1,  6.2. 1.4,  7.0 

5. 5. 2. 3,  5.5.2.17,  6.1.1,  6.3,  6.3.1, 

6.3.2,  6.3.3 

3.2,  3.3,  5.5.1.11,  5.5.2.16,  7.3 

3.1,  6.1,  6.1.2,  6.1.3,  6.3 

3.1,  3.3,  5. 5. 1.2 

5.5.1,  5.5. 2.4,  5.5.2.4.1,  5. 5. 2.4. 2, 

5. 5. 2. 5,  5. 5. 2. 6,  5.5.2.8,  5.5.2.20.1, 

5.5.2.20.3,  6.1.3,  6.2,  6.2. 1.2,  6.3.1, 

6.3.2,  7.3 

5.5. 1.5,  5.5.2.10,  7.4,  7.6 

3.2,  3.3,  5.5. 1.3,  5.5. 1.4,  5.5. 1.5, 

5. 5.1.6,  5.5.1.15,  5. 5. 2.4.1,  5. 5.2.6, 

5.5.2.7,  6.1.1,  6.1.2,  6.1.3,  6.2.1.3, 

6. 2. 1.4,  6.3,  7.3,  7.4,  7.6 
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Computer  Program  Status 
Computer  Program  User  Manual 

Computer  Software  Maintenance  Manual 
Conceptual  Phase 
Configuration  Index 

Configuration  Item 

Conf 1 gurat 1 on  Management 

Configuration  Management  Plan 

Contract 

Contractor  Documents 

Contractor  Document  Formats 
Contractor  Proposal 

Control  Software 
CPCI 

CPDP 

CRISP  •' 

CRWG 

Data  Accession  List 
Data  Base 


6.0,  6.1,  6.1.1,  6.1.2,  6.1.3,  6.1.4,  6.1.5 

3.3,  5.5.1.5,  5.5.2.11,  5.5.2.12,  6.1.4, 


5.5.2.12,  7.6 

3.2,  3.3,  4.2.1,  4.2.2 

3.2,  3.3,  5.5.1.10,  5.5.1.11,  5.5.2.15, 
5.5.2.16,  7.3 

4. 1.5. 2,  4.1.5. 3,  5.5.1,  5.5.2.20.4,  6.1, 

6.1.1 

5.5.1.6,  5.5.1.10,  5.5.2.13,  5.5.2.15, 
6.1.4,  6.2. 1.4,  6.3.2,  6.3.3,  7.3 

3.2,  3.3,  5.5. 1.1,  5.5. 1.9,  5.5.2.14,  6.3, 

6.3.1,  6.3.2,  7.3 

3.2,  3.3,  4.0,  4.2.4,  5.0,  5. 5. 1.2, 

5.5. 1.7,  5.5.2,  5. 5. 2.3,  5.5. 2.5, 

5.5.2.20.3,  6.1.1,  6.2. 1.4 

5.1,  5.2,  5.5,  5.5.1,  5.5.2.20.2,  7.0,  7.1, 
7.3 

5.5.1.15,  5.5.2.20.2 

5.5.1,  5.5. 1.1,  5. 5. 1.2,  5.5. 1.9,  5.5.2.3, 

6.1.1,  6.3,  6.3.2 

3.3,  5.5.2,  5.5.2. 3,  5.5.2.4.1,  5. 5.2.7 

1.4,  4. 1.5. 2,  5.5.1,  5.5.2,  5.5.2.4, 

5. 5. 2.4.1,  5. 5. 2. 4. 2,  5.5.2.8,  5.5.2.13, 

5.5.2.20.4,  6.1,  6.1.1,  6. 2.1. 5 

3.2,  3.3,  4.2.5,  5.5. 1.1,  5.5.1.2,  5.5. 1.4, 
5. 5. 1.9,  5. 5.2.3,  5.5.2. 7,  6.1.1,  6.1.3, 

6.1.4,  6.2,  6. 2. 1.2,  6.3,  6.3.1,  6.3.2, 

7.1,  7.3 

3.2,  3.3,  4.1,  4.1.3,  4.1.4,  4.2,  4.2.3, 
4.2.6,  4.2.7,  7.1 

4.1.3,  4.2.7 

5.4,  5.5. 1.7,  5.5. 1.8,  5.5.1.9,  5.5.2.19 

5. 5. 1.3,  5. 5. 2. 6,  6.2.1.5 


Data  Call 


5.3 


Data  Rights 
DCP 

Design  Phase 
DID 


Document  Evaluation 

Documentation  Needs 
Document  Revision 
Document  Use 


4.1.5.1,  4.2.4,  5.5.2.10,  5.5.2.20.3 

3.3,  4.2.2 

3.1,  6.1,  6.1.2,  6.3 

4.2.4,  5.2,  5.1.1,  5.2,  5.3,  5.5.1.2, 

5. 5. 1.3,  5. 5. 1.4,  5.5. 1.5,  5. 5. 1.6, 

5. 5. 1.7,  5. 5. 1.8,  5. 5. 1.9,  5.5.1.10, 

5.5.1.11,  5.5.1.12,  5.5.1.13,  5.5.1.14, 

5.5.1.15,  5.5.2. 1,  5. 5. 2. 2,  5.5.Z.3, 

5. 5.2.4,  5. 5. 2.4.2,  5.5.2.5,  5.S.2.6, 

5. 5. 2. 7,  5. 5. 2.8,  S.5.2.9,  5.5.2.10, 

5.5.2.11,  5.5.2.12,  5.5.2.13,  5.5.2.14, 

5.5.2.15,  5.5.2.16,  5.5.2.17,  5.5.2.18, 
5.5.2.19,  5.5.2.20.1,  6.2,  6.2.1. 1,  6.2. 1.4 

6.2,  6.2.1,  6.2. 1.1,  6.2. 1.2,  6.2. 1.3, 

6. 2. 1.4,  6.2. 1.5 

3.1 

6.3,  6.3.1,  6.3.2,  6.3.3 

5.2,  7.0 


DSARC 

ECP 

FCA 

FORTRAN 


3.2,  3.3,  4.2.2,  5.3 

3.3,  5. 5. 1.3,  5.5.1.10,  5.5.1.11,  5.5.1.12, 

5.5.2.15,  5.5.2.17,  5.5.2.18,  6.3.2 

5.3,  5. 5. 1.7,  5. 5. 1.8,  5. 5. 1.9,  6.1.4, 
6.1.5,  6.3 

5.5.2.10 


Full-Scale  Development  Phase 

IFPP 

ILSP 

Installation  Phase 
Interface  Description 

ISP 

Life  Cycle  Cost 
Logic  Flows 

Math  Models 


3.2,  3.3 

3.2,  4. 1.5.4,  7.2 

3.3,  4.2,  4.2.3 

3.1,  5.5.2.13,  6.1,  6.1.4,  6.1.5,  6.3 

3.2,  3.3,  5.5. 1.3,  5.S.2.6,  6.1.1,  6.1.2, 

6. 2. 1.5,  6.3,  7.3,  7.4,  7.6 

3.3 

5.2 

5. 5. 1.4,  5.5. 1.6,  5.5.1.15,  5. 5.2. 7, 
5.5.2.13,  6. 2. 1.2,  7.3 

5.5.1.4 
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5.5.2.13,  6.3.3 

3.1,  6.1,  6.3,  6.3.1,  6.3.3 

4.2.5,  5.5. 2.1,  6.1.1 


Operation  ft  Maintenance  Support 
Operation  ft  Support  Phase 
ORLA 
PCA 

PDR 

PMD 

PMP 

Production  Phase 
Program  Design  Language 

Program  Listing 

Programmer  Manual 
Programmer  Notebook 
RFP 

ROC 

SAE  GUIDEBOOKS 
SCN 

SERD 

SFE 

Software  Acquisition  Engineering 
SOU 

Support  Equipment  Plan 
Support  Software 


5.3,  5. 5. 1.3,  5. 5.1.4,  5. 5. 1.5,  5. 5. 1.6, 

5. 5. 2. 7,  5.5.2.11,  5.5.2.13,  6.1.5,  6.2, 

6.3,  6.3.1,  7.3 

5.3,  5. 5. 1.2,  5.5. 1.3,  5.5. 1.4,  5. 5.1. 7, 

5. 5. 2. 3,  5. 5. 2.4,  5.5.2.4.2,  5.5. 2.6, 

5. 5. 2. 8,  6.0,  6.1.1,  6.3,  7.4 

3.2,  3.3,  4.1.2,  4.2.2,  4.2.6,  7.1 

3.2,  3.3,  4.1,  4.1.3,  4.2.2,  4.2.6,  4.2.7 

7.1 

3.2,  3.3 

5. 5. 1.4,  5. 5. 1.5,  5.S.2.7,  5.5.2.11, 

6.2.1.2 

5. 5. 1.4,  5.5. 1.6,  5. 5. 2.4.1,  S.5.2.7, 
5.5.2.13,  6.1.3,  6.1.4,  7.3 

3.2,  5. 5.1.5,  5.5.2.10 

5.5.1.5 

3.2,  3.3,  4.1,  4.1.5,  4.2,  4.2.4,  5.5. 1.1 

7.2 

3.2,  3.3,  4.1,  4.1.1,  4.2.1,  7.1 

1.2 

3.2,  3.3,  5.5.1.10,  5.5.1.11,  5.5.2.15, 
5.5.2.18,  6.3.2,  7.3 

3.3,  4.2.5,  4.2.6,  5.5.2,  5. 5.2.1,  5.5.2.J 

6.1.1 

3.2,  4. 1.5.6,  4.2.7,  7.2 
1.4.1 

3.2,  3.3,  4. 1.5.1,  4.2,  4.2.4,  4.2.5,  5.3 

5. 5. 1.2,  5.5. 1.9,  5. 5. 2. 3,  5.5.2.14,  7.2 

3.3,  4.2.5,  5.5.2,  5.5.2.1 

3.3,  5.5.2,  5. 5. 2. 3,  5.5.2.4.1,  5. 5. 2. 7 
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Tailoring  DIDs 

Technical  Order 
TDSP 

TECPD 

Test  and  Integration  Phase 
Testing 
Test  Report 
Test  Software 

Test  Plans/Procedures 
TRD 

TS  Document  Sequence 
TS  System  Spec 


5.1.3,  5.2,  5. 5. 1.2,  5. 5. 1.3,  5. 5. 2.3, 

5. 5. 2. 4. 2,  5. 5.2.5,  5.5.2.6,  5. 5.2. 7, 
5.5.2.11,  5.5.2.12,  6.2 

5.5.2.20,  5.2.2.20.4,  7.6 

5.5. 1.4,  5.5. 1.5,  5.5.1.15,  5.5. 2.7, 
5.5.2.11 

3.2,  5. 5. 1.5,  5.5.1.15,  6.3,  7.6 

6.1,  6.1.3,  6.1.4,  6. 2. 1.2,  6.3,  6.3.1 
5. 5. 1.7,  5. 5.2.8,  6.2. 1.3 

3.2,  3.3,  5. 5. 1.8,  5. 5. 1.9,  6.1.4,  6.3,  7.5 

3.3,  5.5.2,  5.5. 2. 3,  5. 5. 2. 4.2,  5. 5.2. 5, 

5. 5. 2.6,  5. 5. 2.7,  5.5.2.8,  5.5.2.10, 
5.5.2.20.4 

3.2,  3.3,  5. 5. 1.6,  5.5.2. 8,  6.1.3,  6.3,  7.6 

3.3,  5. 5. 2.1,  5.5.2. 4.2,  5.5.2.5,  5. 5. 2.6, 

5.5.2. 7,  5.5.2.11,  5.5.2.20.1,  5.5.2.20.3, 
6.1.1,  6.1.2,  7.4 

3.2 

4. 1.5. 3,  5.5.1,  5. 5. 1.7,  6.1.1,  6.3,  6.3.1, 

7.2 


User  Need 
Unique  DIDs 
Validation  Phase 
VDD 

WBS 


5.1.2,  5.1.3, 

5.1.3,  5.2, 

3.2,  3.3,  4.2.7 

3.2,  3.3,  5. 5. 1.6, 

7.3,  7.6 

4. 1.5.1 


5. 5. 2. 6 

5.5.2.13,  6.3,  6.3.1 


6.2. 1.4 
5. 5. 1.3, 
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